System, method and apparatus for monitoring picking activities

ABSTRACT

A system, a wearable device and method for monitoring picking activities. The method comprises: obtaining a visual input from a sensor located on a wearable device, wherein the wearable device is worn on a hand of a picker, wherein the sensor is configured to capture at least an interior portion of a hand of the picker, wherein the picker is tasked with picking items to fulfill a shopping order of a customer, wherein the shopping order comprises a list of items; determining, based on the visual input, that the picker picked up an item and placed the item in a tote associated with the shopping order of the customer, wherein said determining that the picker picked up the item and placed the item comprises identifying the item; and in response to said determining: performing a fulfillment-related action.

CROSS-REFERENCE TO RELATED APPLICATIONS

This application is a continuation of and claims the benefit of U.S.patent application Ser. No. 16/881,105, entitled “HAND ACTIONS MONIORINGDEVICE”. Filed May 22, 2020, which claims the benefit of U.S.provisional Application No. 62/876,995, entitled “Smart Retail Device”,filed Jul. 22, 2019, all of which are hereby incorporated by referencein their entirety, without giving rise to disavowment.

TECHNICAL FIELD

The present disclosure relates to wearable devices in general, and towearable device for monitoring hand actions, in particular.

BACKGROUND

AMAZON GO™ is a chain or partially automated stored, where customers areable to purchase products without being check out by a cashier or usinga self-checkout station. Wikipedia details that according to apromotional video published by Amazon, the store concept uses severaltechnologies, including computer vision, deep learning algorithms, andsensor fusion to automate much of the purchase, checkout, and paymentsteps associated with a retail transaction. The store concept is seen asa revolutionary model that relies on the prevalence of smartphones andgeofencing technology to streamline the customer experience, as well assupply chain and inventory management. However, public rollout of theSeattle Amazon Go prototype location was delayed due to issues with thesensors' ability to track multiple users or objects within the store,such as when children move items to other shelves or when more than onecustomer has a similar body habitus.”

The AMAZON GO™ store relies on an array of in-store sensors. The ceilingof the store has multiple cameras and store shelves have weight sensors,to detect which item a customer took. If a customer takes an item offthe shelf, it will be added to the customer's virtual cart. Similarly,if a customer places an item back on the shelf, it is removed from thecustomer's virtual cart.

BRIEF SUMMARY

One exemplary embodiment of the disclosed subject matter is a methodcomprising: obtaining a visual input from a sensor located on a wearabledevice. The wearable device is worn by a subject. The sensor isconfigured to capture at least an interior portion of a hand of thesubject. The method further comprises: identifying, based on the visualinput, an action performed by the subject using the hand of the subject,wherein the action is performed with respect to an object, wherein saididentifying comprises identifying the object. The method furthercomprises: performing a responsive action based on at least one of theaction and the object.

Optionally, the method further comprises: in response to detecting acheck-in activity being performed, continuously monitoring the subject.Said continuously monitoring the subject comprises said obtaining thevisual input. Said continuously monitoring is terminated upon detectinga check-out activity being performed.

Optionally, the interior portion of the hand comprises a distal portionof a palm, whereby the visual input captures at least a portion of theobject when the object is being held by the hand.

Optionally, the sensor is configured to capture an item grasped byfingers of the subject during grasping thereof.

Optionally, the wearable device is configured to be worn on a wrist ofthe subject, whereby positioning the sensor to face a palm of the hand.

Optionally, a view of the sensor is blocked, at least in part, by thehand.

Optionally, at least 5% of the visual input comprises a view of theinterior portion of the hand.

Optionally, the action performed by the subject comprises at least oneof: touching the object using the hand; picking up the object using thehand; releasing the object after being held by the hand; pressing on theobject using the hand; moving, using the hand, the object from a firstlocation to a second location; covering the object using the hand; andunwrapping the object using the hand.

Optionally, the wearable device is utilized for a self-service shoppingof the subject in a store, whereby enabling the subject to shop in thestore without manually scanning purchased items before leaving thestore; wherein the object is an item for sale in the store; wherein saididentifying comprises identifying that the subject picks up the item forsale; wherein the responsive action comprises: updating a virtual cartof the subject to include the item for sale; wherein the method furthercomprises: identifying a check-in activity of the subject, whereby thesubject commencing a self-service shopping session at a store, whereinsaid identifying is performed during the self-service shopping session;and in response to identifying a check-out activity of the subject,performing a transaction based on content of the virtual cart of thesubject.

Optionally, the method further comprises: obtaining a second visualinput from the sensor; determining, based on the second visual input,that the subject has decided not to purchase the item for sale; and inresponse to said determining, updating the virtual cart of the subjectto exclude the item for sale.

Optionally, the responsive action further comprises emitting an auditorycue indicating the addition of the item for sale to the virtual cart.

Optionally, the method further comprises: displaying to the subject acontent of the virtual cart.

Optionally, the method further comprises: during the self-serviceshopping session, detecting a tampering event aimed at interfering withmonitoring of the hand of the subject using the wearable device; and inresponse to said detecting the tampering event, performing ananti-tampering action.

Optionally, said wearable device is configured to perform said detectingonly during the self-service shopping session, wherein the tamperingevent comprises at least one of: removal of the wearable device andblocking view of the sensor; whereby the subject is able to perform thetampering event after the self-service shopping session ends withoutresulting in the anti-tampering action.

Optionally, said detecting comprises obtaining readings from a motionsensor of the wearable device to determine a behavioral pattern of thesubject that is indicative of an attempt to prevent monitoring of thehand of the subject.

Optionally, the method further comprises: utilizing the wearable deviceto remove a theft detection tag coupled with the item for sale.

Optionally, said identifying comprises identifying the item for salebased on the visual input and based on a positioning reading of thewearable device.

Optionally, the method further comprises: determining, based on thepositioning reading of the wearable device, a subset of a catalog ofitems; and performing product recognition to identify the item for sale,wherein the product recognition is performed with respect to the subsetof the catalog of items.

Optionally, the method further comprises: obtaining a second visualinput from the sensor, wherein the second visual input captures acontent of a physical tote; determining, based on the virtual cart andon the second visual input, a discrepancy between content of the virtualcart and content of the physical tote; and in response to saiddetermining, performing a second responsive action.

Optionally, the second responsive action comprises at least one of:marking the virtual cart as invalidated and updating the virtual cartbased on the second visual input.

Optionally, the wearable device is utilized for manual fulfillment of ashopping order of a customer, wherein the shopping order comprises alist of items; wherein the subject is a picker picking items to fulfillthe shopping order of the customer; wherein the action comprises pickingup the object and placing the object in a tote associated with theshopping order of the customer; and wherein the responsive actioncomprises: identifying a corresponding item to the object in the list ofitems; and marking the corresponding item as fulfilled.

Optionally, the method further comprises: obtaining a second shoppingorder of a second customer, wherein the wearable device is configured tomonitor the picker while the picker fulfills the second shopping order.

Optionally, the wearable device is utilized for manual fulfillment of ashopping order of a customer, wherein the shopping order comprises alist of items; wherein the subject is a picker picking items to fulfillthe shopping order of the customer; wherein the action comprises pickingup the object and placing the object in a tote associated with theshopping order of the customer; and wherein the responsive actioncomprises: identifying a mismatch between the object and the list ofitems; and alerting the subject of the mismatch.

Optionally, the action comprises placing the object; wherein theresponsive action comprises: determining a geospatial location of theobject after being placed, and updating a mapping of items in a facilityto indicate the object is located at the geospatial location.

Optionally, the wearable device comprises a first wearable component anda second wearable component, wherein the first wearable component isconfigured to be worn on the hand of the subject, wherein the secondwearable component is configured to be worn on a second hand of thesubject, wherein the sensor comprises a first sensing component locatedon the first wearable component and a second first sensing componentlocated on the second wearable component, wherein the visual inputcomprises input from the first sensing component and from the secondsensing component.

Optionally, the responsive action comprises: comparing the action with arule; and in response to a violation of the rule, issuing a safetyalert.

Optionally, the rule is related to an administration of medicine to apatient, wherein the subject is administering medicine to the patient,wherein the object is involved in the administration of the medicine,wherein the violation of the rule comprises at least one of:administering a wrong dosage, administering a wrong drug, performing awrong procedure, administering the medicine to a wrong patient.

Optionally, the wearable device is utilized for monitoring a health-caresystem, wherein the method further comprises: wherein the subject is ahealth-care worker, wherein the wearable device is configured tocontinuously monitor the hand of the subject during treatment ofpatients; wherein the action comprises a treatment action on a patient.

Optionally, the wearable device is utilized for managing a manufacturingplant, wherein the method further comprises: wherein the subject is anemployee on the manufacturing plant, wherein the wearable device isconfigured to continuously monitor the hand of the subject during a workshift of the subject in the manufacturing plant; wherein the actionrelating to at least one of: an assembly of a product, wherein theobject is a component in the product; an assembly of a product, whereinthe object is a tool utilized for the assembly of the product; operatinga machine, wherein the object is a component of the machine; andoperating a machine, wherein the object is a component of a productprocessed by the machine.

Another exemplary embodiment of the disclosed subject matter is awearable device, wherein the wearable device is configured to be worn ona hand of a user, wherein the wearable device comprises: a visual sensorconfigured to continuously capture an interior portion of the hand,wherein the wearable device is configured to provide images captured bythe visual sensor to be utilized to identify an action performed by thehand of the user and to identify an object upon which the action isperformed; and a communication unit configured to connect the wearabledevice to a controller external to the wearable device.

Optionally, activation and de-activation of the wearable device isperformed automatically by the controller.

Optionally, the controller is configured to determine a responsiveaction based on the action or the item.

Optionally, the wearable device is configured to be utilized forself-service shopping, wherein the wearable device is configured to beutilized to identify items grabbed by the hand and moved to or from aphysical shopping tote of the user, wherein the items are identifiablebased on input of the visual sensor.

Optionally, the wearable device is configured to be associated with avirtual cart upon initiating a self-shopping session, wherein thevirtual cart indicates a list of items shopped by the user, wherein thevirtual cart is automatically updated based on items moved to and fromthe shopping cart.

Optionally, the wearable device is configured to be utilized for manualfulfillment of a shopping order of a customer, wherein the shoppingorder comprises a list of items; wherein the user is a picker taskedwith picking items to fulfill the shopping order of the customer;wherein the action comprises picking up the object and placing theobject in a tote associated with the shopping order of the customer.

Optionally, the wearable device further comprises a tampering detectionmodule, wherein said tampering detection module is configured to monitorand detect a tamper event during a shopping session of the user, whereinsaid tampering detection module is configured to avoid monitoring useractivity outside the shopping session.

Optionally, the wearable device is a wristband worn on a wrist of theuser.

Yet another exemplary embodiment of the disclosed subject matter is asystem for a self-service shopping in a store, comprising: a pluralityof wearable devices; a control module; a catalog database retainingvisual representations of items in the store; and a payment module;wherein each of said plurality of wearable devices is being worn on ahand of a different user and configured to obtain visual input ofactivity performed by the hand of the different user; wherein saidcontrol module is configured to identify an action made by a user andcaptured by a wearable device worn by the user, wherein the action isassociated with the user modifying content of a physical shopping tote,wherein said control module is configured to update a virtual cart ofthe user in response to the identification of the action by the user,wherein the virtual cart is updated based on an identification of theitem from said catalog database; wherein said control module isconfigured to invoke said payment module in response to a check-outactivity of the user; and wherein said payment module is configured toperform a transaction based on a list of shopped items of the user asindicated in the virtual cart.

Optionally, the system further comprises: an activation moduleconfigured to automatically perform activation and de-activation of thewearable device, wherein said activation module is configured toautomatically activate the wearable device in response to identifying acheck-in activity associated with the user, wherein the wearable deviceis configured to continuously monitor actions of the user between thecheck-in activity and the check-out activity.

Optionally, the wearable device comprises a visual sensor configured tocontinuously capture an interior portion of the hand, wherein the visualinput provided by said wearable device module comprises images capturedby the visual sensor.

Optionally, said control module is configured to automatically updatethe virtual cart of the user during the self-shopping session, whereinthe update of the virtual cart comprises: automatically adding items tothe virtual cart based on items picked up and put into the physicalshopping tote of the user, and automatically removing items from thevirtual cart based on items removed from the physical shopping tote ofthe user.

Optionally, the system further comprises: an output interface to beutilized by the user, wherein the output interface is configured todisplay to the user a content of the virtual cart and to provide anindication of an addition of an item to or removal of an item from thevirtual cart.

Optionally, said control module is configured to obtain visual input ofthe content of the physical shopping tote; wherein said control moduleis configured to determine, based on the virtual cart and on the visualinput of the content of the physical shopping tote, a discrepancybetween content of the virtual cart and the content of the physicalshopping tote; and wherein said control module is configured to performa responsive action in response to a determined discrepancy, wherein theresponsive action comprises at least one of: marking the virtual cart asinvalidated and updating the virtual cart based on the visual input ofthe content of the physical shopping tote.

Optionally, said control module is configured to identify a placementlocation of an object, whereby determining a mapping of geo-spatiallocations of items in the store.

Optionally, the system further comprises: a tampering detection module,wherein said tampering detection module is configured to monitor anddetect a tampering event during the self-shopping session of the user,wherein the tampering event is an event aimed at interfering withmonitoring of the hand of the user using the wearable device, whereinsaid tampering detection module is configured to avoid monitoring useractivity outside the shopping session; wherein said control module isconfigured to perform an anti-tampering action in response to detectingthe tampering event.

Optionally, the store is a crowded store, having a ratio between numberof people in the store and a floor size of the store above 1:10.

Yet another exemplary embodiment of the disclosed subject matter is asystem for a fulfillment of shopping orders in a fulfillment center,comprising: a plurality of wearable devices; a control module; a catalogdatabase retaining visual representations of items in the store; and apayment module. Each of said plurality of wearable devices is being wornon a hand of a different picker and configured to obtain visual input ofactivity performed by the hand of the different picker. Each picker istasked with picking items to fulfill a shopping order of a differentcustomer. Said control module is configured to identify a picking actionthat comprises picking up the object and placing the object in a toteassociated with the shopping order of the customer, wherein the pickingaction is captured by the wearable device. Said control module isconfigured to identify a corresponding item to the item in the shoppingorder and mark the corresponding item as fulfilled, wherein theidentification of the corresponding item is based on said catalogdatabase. Said control module is configured to invoke said paymentmodule in response to a determination that the shopping order isfulfilled.

THE BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

The present disclosed subject matter will be understood and appreciatedmore fully from the following detailed description taken in conjunctionwith the drawings in which corresponding or like numerals or charactersindicate corresponding or like components. Unless indicated otherwise,the drawings provide exemplary embodiments or aspects of the disclosureand do not limit the scope of the disclosure. In the drawings:

FIGS. 1A-1D show schematic illustrations of a hand action monitoringwearable device, in accordance with some exemplary embodiments of thedisclosed subject matter;

FIG. 2 shows a schematic illustration of an exemplary self-serviceshopping scheme, in accordance with some exemplary embodiments of thedisclosed subject matter;

FIG. 3 shows schematic illustrations of visual inputs provided by a handaction monitoring wearable device, in accordance with some exemplaryembodiments of the disclosed subject matter;

FIGS. 4A-4C show schematic illustrations of exemplary environments inwhich the disclosed subject matter may be utilized, in accordance withsome exemplary embodiments of the disclosed subject matter;

FIGS. 5A-5C show flowchart diagrams of methods, in accordance with someexemplary embodiments of the disclosed subject matter;

FIGS. 6A-6B show flowchart diagrams of methods, in accordance with someexemplary embodiments of the disclosed subject matter; and

FIG. 7 shows a block diagram of a system, in accordance with someexemplary embodiments of the disclosed subject matter.

DETAILED DESCRIPTION

One technical problem dealt with by the disclosed subject matter is toenable monitoring actions performed by a human subject without affectingperformance of such actions. It may be desired to monitor such actionswithout violating privacy of the human subject or others in the humansubject's environment. The actions may be natural actions, incidentalactions, intentional actions, or the like. The actions may be performedby the hand of the human subject, using instruments, tools, or the like.The actions may be performed on objects, on other subjects, or the like.

It is noted that monitoring hand actions may be essential in variousdisciplines, starting from safety inspection such as in factories, toautomating actions such as in self-service shopping. As an example, inself-service shopping, a shopper can enter a store (such as asupermarket, grocery store, fashion store, warehouse, or the like),collect items using her hands and perform other related actions. Assuch, the shopper may purchase products without being check out by acashier, without requiring additional scanning of the products beforeexisting the store, or the like.

In some exemplary embodiments, different actions of human subjects maybe required to be continuously monitored, in order to maintain safetyrequirements, to supervise the actions, inspect the actions, or thelike. As an example, the action of providing drugs to patients performedby health-care staff, may be desired to be monitored and supervised, inorder to verify that a correct dose was provided for a patient, toregister the timing of providing the drug, to coordinate between theactions performed by different staff members, such as different nursesin different shifts, observing side effects, or the like.

In some exemplary embodiments, actions may be monitored using sensors,such as cameras, radars, LiDARs, or the like, that observe the humansubject. The observing sensors may be located within the environment ofthe human subject or the location the actions are supposed to beperformed. In addition to requiring special equipment and highadditional costs, such monitoring may be limited due to limited abilityof capturing the actions from different angles, limited ability oftracking the human subject in different locations, limited ability oftracking multiple subjects simultaneously, limited ability in crowdedstores or in situations where customers block the view of othercustomers, or the like. Additionally or alternatively, monitoring theactions using sensors observing the human subject may affect the privacyof the human subject and other subjects in her environment.Specifically, monitoring from afar the human subject can also captureand monitor the subject's private encounters, conversations, or thelike. Refereeing to the above mentioned example, monitoring health-carestaff may not only violate the privacy of the health-care staff members,but also patients treated thereby.

Another technical problem dealt with by the disclosed subject matter isto provide for a solution that enables a self-service shopping, withoutthe need of expensive additional equipment or changing the structures ofthe store from the retailer side, and without requiring additionalscanning of purchased items from the customer side. Basic self-serviceshopping solutions may utilize self-checkout machines as an alternativeto the traditional cashier-staffed checkout. However, the customer maybe still required to stop for check out and even to perform the job ofthe cashier themselves, by scanning and applying payment for shoppeditems. Besides being uncomfortable, more complex, and time and effortconsuming from the customer side, such solutions may have multipledisadvantages for retailers, such as being vulnerable to shoplifting,mistaken self-checkouts, having partial visibility, more prone tofailure, reducing the possibilities for customers and store staff tointeract, and adversely affecting customer service in general. Othersolutions may enable the customer to purposely scan each product whenbeing picked, e.g., using an on-cart barcode reader, instead of scanningat a self-service cashier. However, even such solutions may beassociated with different problems, such as spending time in scanning,being susceptible to shoplifting, multiple scanning errors or inaccuratescanning, or the like.

In some exemplary embodiments, solutions for self-service shopping maybe required to automate much of the purchase, checkout, payment stepsassociated with a retail transaction, or the like. In some exemplaryembodiments, some solutions may use a set of cameras and shelvesequipped with weight sensors that are configured to identify whichproducts the shopper collects and automatically calculate the bill. Suchsolutions may require special equipment to be deployed in the store andmay not be suitable for Small-Medium Business (SMBs) in view of the highcosts, may require the store being in a predetermined structure with nodesign flexibility, may be configured to work only in clean environmentswhere different objects should be removed (such as signs, ads, or thelike), or the like.

Besides requiring special equipment and high additional costs, suchsolutions may be less conformed to the market needs due to issues withthe sensors' ability to track multiple users or objects within thestore, or additional natural habits of shoppers, such as when childrenmove items to other shelves, when more than one customer has a similarbody habitus, cannot be employed in crowded stores, or the like. As anexample, other vision-based potential solutions that rely on peripheralscanners, stationary scanners, or the like, and user-tracking mayfunction only with a limited number of shoppers inside the store, suchas 1 shopper to a 100 square meters, 5 shoppers to 100 square meters, orthe like. A larger number of shoppers may cause a breakdown of suchshopper-tracking technologies, as people become too difficult to follow,may have trouble tracking products that are moved from their proper spoton a shelf, view of a customer's activity may be blocked due to othercustomers in the same area, or the like. Additionally or alternatively,such solutions may be based on the assumption that every item is locatedin its place at the specific shelf. If a customer takes an item off theshelf, it will be added to the person's virtual cart. If a customerplaces an item back on the shelf, it is also removed from the customer'svirtual cart. Accordingly, if a customer returns an item back to adifferent location within the store, a confusion may occur, if anothercustomer than picks the same item from that location. Additionally oralternatively, such solutions may have other disadvantages, such asaffecting privacy of the consumer, enabling a limited number ofsimultaneous consumers, or the like.

Yet another technical problem dealt with the disclosed subject matter isidentifying and preventing tampering events during self-serviceshopping. One of the challenging problems associated with self-serviceshopping, is that in the lack of human monitoring, customers may attemptto rig the system and commit shoplifting. As an example, shoppers mayattempt to concealing items from cameras or visual sensors, avoidscanning certain items, or the like. In some exemplary embodiments,self-service shopping solutions may be highly exposed to shopliftingevents for multiple reasons, such as that self-checkout theft may beeasier to get away with, self-service shopping may be associated withless attention from both employees and other customers, the customer mayclaim that items taken accidentally or inadvertently, psychologicaleffects such as lack of guilt during to interaction with a machinerather than a human being, lower chances to prosecute, or the like.Self-service shopping solution may be required to detect and alertattempts to steal, to prevent implementation of such attempts, or thelike.

Yet another technical problem dealt with the disclosed subject matter isto enable fulfillment of home shopping by human pickers. In someexemplary embodiments, the picker may be tasked with picking itemsselected by the customer and prepare them for shipment. The picker maypick the items in the store itself, in a storage, in a warehouse, or thelike. The picker may be required to read digital or printed orders,including a list of items, pick up the items, place the items in thecorrect tote associated with each order, transfer the totes fordelivery, or the like. Current models of home shopping may involvedisadvantages for both the retailers and the customers. On the one hand,the customer may not be able to dynamically update her shopping list, toinspect products before being shipped to her, wrong items may beincluded in the order and require for the customer's review of thesupplied items as well as to contact or the like. On the other hand, theretailer cannot provide an updated items list at each time point for thecustomer, wrong items may be picked and reduce cost-effectiveness forthe retailer, or the like. Additionally or alternatively, pickers maynot be able to efficiently fulfill more than one order simultaneously,due to potential mix-ups and confusions.

Yet another technical problem dealt with the disclosed subject matter ishandling theft detectors during self-service shopping. In some exemplaryembodiments, different retail stores, such as fashion store, libraries,or the like, may utilize electronic article surveillance systems toprevent shoplifting. A theft detector, such as electro-magnetic tags,radio frequency tags, or the like, may be fixed to merchandise andremoved or deactivated by the clerks when the item is bought or checkedout. However, such systems may force the customer to stop at thecheckout in order to remove or deactivate the theft detectors from theitems, even during self-service shopping.

Yet another technical problem dealt with the disclosed subject matter isto determine location of objects in changing environments, such asstores, pharmacies, medical centers, factories, fulfillment centers,warehouses, or the like. In some exemplary embodiments, it may beimportant to determine the exact location of objects, such as forstocktaking, inventory counting, efficiently finding objects, insuringsafety, or the like. In some exemplary embodiments, the location ofobjects may dynamically change, such as due to inadvertent locationchange by a staff worker, customers placing objects in a non-designatedlocation, human error such as caused by working under pressure, or thelike. Determining the location may be critical in such cases and othercases also for safety verification.

One technical solution is to track hand actions using a wearable smartdevice worn by a human subject. The wearable smart device may be worn onthe hand of the human subject, on the wrist of the human subject, or thelike. The wearable smart device may be equipped with a vision sensor,such as a digital camera, a radar sensor, a radio waves based sensor, alaser scanning sensor, a LiDAR, an Infrared (IR) sensor, an ultrasonictransducer, or the like. The vision sensor may be configured to observethe interior portion of the hand of the human subject, other portions ofthe hand of the human subject, an area surrounding the hand of the humansubject, a 360 degree view of the hand, or the like. Based on visualinput provided by the wearable smart device, the object grasped by thehand, and the action performed thereon, may be identified, and aresponsive action may be performed accordingly.

In some exemplary embodiments, the wearable smart device may be a devicethat can be worn on human's hand without affecting actions performed bythe hand, such as a bracelet, a wristband, a watch, or the like.However, other hand wearables may be utilized such as a glove thatcovers all or part of the hand (e.g., a few fingers, a finger cover), orthe like. Additionally or alternatively, the wearable smart device maybe embedded in a smart watch or other wearable device of the humansubject being observed. The wearable smart device may be worn in asingle hand, in both hands separately, in both hands simultaneously, orthe like.

It is noted that in some embodiments, a body-worn device that is notworn on the hand may be utilized. As an example, the device may be wornon a collar, on a torso, or the like, while having view of the hand.Such embodiments may be less reliable than hand-worn devices, and may besusceptible to manipulations by the subject. In case of pickers andother staff members, such devices may be utilized when considering thatthe subject is considered reliable.

In some exemplary embodiments, the wearable smart device may beconfigured to identify when the hand of the human subject touches anitem, picks the item up, moves the item from one location to another,releases the item, places the item, pressing on the item, unwrapping theitem, or the like. The device may be further configured to identify theitem being touched by the hand(s), such as the type of the item, shape,name, or the like. Additionally or alternatively, the wearable smartdevice may be configured to identify other attributes related to theitem, such as weight, size, temperature, texture, or the like. In someexemplary embodiments, the wearable smart device may be configured toidentify when the action that the hand(s) perform on an item (e.g.,touches, holds, grabs, releases, or the like) based on the visual inputprovided by the vision sensor. The wearable smart device may beconfigured to identify the item using the visual input. As an example,the device may be configured to identify the item based on an opticalimage, based on QR code, barcode, any combination of letters, numbers orimages, or the like. In some exemplary embodiments, computer visiontechniques may be employed to analyze the images. The image analysis maybe performed on-device. Additionally or alternatively, off-deviceanalysis may be implemented, such as to preserve battery and reducecomputation requirements from the device, or the like.

In some exemplary embodiments, the wearable smart device may beconfigured to identify in real-time, a target destination in which theobject is being placed in during the shopping, such as a bag, a shoppingcart, the hands of the subject, or the like. The visual input of thewearable smart device may be analyzed to identify a surface on which theobject is being placed, a predetermined shopping cart such as a physicalshopping cart of the store, a personal shopping bag, or the like. Insome exemplary embodiments, the wearable smart device may be configuredto identify a candidate shopping cart, and validate such candidate to bethe shopping cart during the shopping session, such as based onadditional visual input, such as later visual input, or the like. As anexample, in response to identifying a container in which a first objectis placed on, the container may be determined to be a candidate shoppingcart. In response to identifying that other objects are being placed inthe container by the subject, the container may be validated as theshopping cart. As another example, the container may be determined to bea component in the shopping carts, such as a bag placed inside theshopping carts, a section in a divided shopping cart, or the like. Asyet another example, the target destination may be the customer's hands,which may be considered a container. As yet another example, thecontainer may be determined as a temporary shopping cart upon movingobjects placed therein to a different shopping cart. In some exemplaryembodiments, the identified shopping cart may be container that isunique and was never before used as a shopping cart. For example, thecustomer's personal bag may be a unique bag that was never encounteredbefore. The shopping cart may be identified by first identifying it as acandidate due to it being a container. In view of repetitive insertionof items to the candidate, in view of the candidate being in the view ofthe sensor during the shopping session over time, or in view of otherconsiderations, the shopping cart candidate may be validated. It may benoted that the shopping cart may dynamically change during the shoppingsession, such as because of placing objects in different containersduring the shopping session, keeping the objects in the hands for awhile before being placed in the shopping cart, or the like. A virtualshopping cart may be updated to be associated with each determinedshopping cart utilized during the shopping session. An additionalvalidation may be performed to determine that the virtual cart comprisesa correct combination of items purchased by the subject and placed indifferent containers.

In some exemplary embodiments, the one or more sensors may befunctionally coupled to a learning module, such as implemented usingArtificial Neural Networks (ANN), supervised learning, or the like,configured to analyze photos captured by the sensors and inferinformation therefrom. Additionally or alternatively, the wearable smartdevice may comprise a chip or another hardware technology configured toreceive, collet and process pictures, videos signals, or the like ascaptured from the sensors of the wearable smart device. Additionally oralternatively, the retail smart wristband may comprise a transmitterutilized for transmitting input captured by the sensors of the wearablesmart device to a backend solution. Such transmitter may use Wi-Fi,Bluetooth, or the like for transmitting the data.

In some exemplary embodiments, the subject may be required to wear twowearable smart devices, one on each hand, in order to monitor actions ofthe two hands of the subjects. In some exemplary embodiments, a singlewearable device that comprises two components that can be disconnectedmay be utilized. The single wearable device may be worn by the subjecton one hand, or can be divided into the two components each of which maybe worn on a different hand. Each component may be associated with asensor that may be configured to continuously monitor the respectivehand and provide visual input thereof.

Another technical solution is utilizing the wearable smart device forretail uses. The wearable smart device may be worn by a customer toidentify items being collected by the customer. In some exemplaryembodiments, the wearable smart device may be configured to identifywhen the customer's hand touches an item, picks the item up, moves theitem, releases the item, places the item on a shopping bag or a shoppingcart, places the item back to a location in the store, or the like. Thedevice may be configured to identify a shopping cart in which purchaseditems are being placed therein, such as based on analysis of thecustomer's environment, analysis of the customer's behavior, analysis ofmovement patterns of the customer, or the like. The device may befurther configured to identify the item being touched by the customers'hand(s), such as the type of the item, shape, name, or the like.Additionally or alternatively, the device may be configured to identifyother attributes related to the item, such as weight, number of units inthe package, size, expiry date, or the like. The device may be personal,e.g., a property of the customer that may be adapted to fit to any storethat the customer enters, or a property of the store, that may beprovided to each customer that enters the store and personalized orpaired to the customer accordingly.

In some exemplary embodiments, when a customer enters a store, thecustomer may put the wearable smart device on. The wearable smart devicemay be automatically or manually synchronized to the mobile phone of thecustomer, to identify the store (e.g., in case of the device belongingto the customer) or the customer (e.g., in case of the device beingproperty of the store). The wearable smart device may be configured toidentify any item that the customer picks up and puts into the shoppingbag or cart, keeps it within his hand, moves to the other hand, put in adifferent shopping bag, or the like. A digital shopping list may beautomatically created, updated and checked without the need for stoppingat checkout point, or being re-reviewed an item by an item, or the like.

Additionally, or alternatively, when a customer enters a store, thecustomer may receive a container, herein referred to as a shopping cart,that connects to the wearable smart device, or is otherwise associatedtherewith. The shopping cart may comprise a wireless communicationmodule enabling direct communication with the device (e.g., viaBluetooth), indirect communication therewith (e.g., via a Wi-Fi, via theInternet, or the like), or the like. Additionally or alternatively, theshopping cart may comprise a screen that interfaces to the wearablesmart device and displays the items being added to the cart, the totalcost of the shopped items, or any other information related to theitems. The shopping cart interface may be utilized to verify that theitem that the customer is being charged is entered into the correctbill, e.g., to prevent a situation that the customer is being chargedfor an item not inserted into the shopping cart (as an example, when thecustomer helps another shopper, picks an item from the floor, picks anitem from a different cart, enters the item to a wrong cart, or thelike). Additionally or alternatively, a mobile application may beutilized to provide a similar display to the user.

In some exemplary embodiments, the wearable smart device may beconfigured to identify when the customer's hand(s) perform an action onan item (e.g., touches, holds, grabs, releases, or the like) based ontouch contact between the device and the item, the device and otheritems, or the like. As an example, the device may comprise pressuresensors that may be located on certain locations thereof that may be incontact with the items, such as three fingers (e.g., on the thumb, indexfinger and middle finger), or the like. The device may identify, usingthe pressure sensors that the customer's hand holds an item, releasesthe item, or the like. For example, existence of pressure may beindicative of the customer holding the item, while lack of pressure mayindicate that the item is released. Additionally or alternatively, thedevice may comprise accelerometer sensors, configured to identify andmonitor the movement of the hand. The device may be configured toidentify that the item is being held, moves, inserted into the cart, orthe like, based on the movement pattern of the customer's hand. It maybe noted that the pressure sensors or the accelerometer sensors may beutilized for determining additional attributes of the item, such asweighing using the pressure sensors, size and number using theaccelerometer sensors, or the like. Additional sensors may be utilized,such as, temperature sensors, scanners, or the like

In some exemplary embodiments, based on the visual input, the item andthe action being performed thereon may be identified. The device maycomprise a vision sensor such as a camera, an IR sensor, a radar, aLiDAR, an ultrasonic transductor, electro-magnetic waves-based sensor, alaser-based sensor, a visible-light based sensor, an invisiblelight-based sensor, a combination thereof, or the like, that providesvisual representation of the item. It is noted that the visual input maybe a visual representation as would be captured by a human being, datathat is computationally processed to generate an image (e.g., by animaging process), or the like. The device may be configured to identifythe item using the visual data. As an example, the device may beconfigured to identify the item based on an optical image, based on QRcode, barcode, an identifier of the item, such as a combination ofnumbers, letters, icons, or the like. In some exemplary embodiments,computer vision techniques may be employed to analyze the images, suchas object recognition techniques, image analysis techniques, machinelearning, or the like. In some exemplary embodiments, the image analysismay be performed on-device. Additionally or alternatively, off-deviceanalysis may be performed to preserve battery and reduce computationrequirements from the device.

In some exemplary embodiments, the device may comprise location sensorsthat may be configured to identify the item and the action based on thelocation of the item. The exact location of the item may be determinedby a triangulation of known locations inside the store, using a threedimensional mapping of the store shelves, or the like. In some exemplaryembodiments, location sensors may be Radio Frequency (RF)-based, may bebased on GPS, may be based on cellular towers, indoor-beacons, or thelike. In some exemplary embodiments, the location sensors may be adaptedfor indoor usage, such as may be based on triangulation of signalsprovided within the store by designated hardware. Additionally oralternatively, tagging techniques may be utilized to identify an item.As an example, a Radio Frequency Identification (RFID) sensor may beutilized to read RFID information from a tag embedded in or coupled tothe item. The RFID information may comprise an indication of theidentity of the tagged item, such as a barcode, a sequence of letters,numbers, icons, or the like. As another example, a location of the itemmay be identified based on a recognition of shelves inside the store,such as based on an identifier of a shelf the item is located thereon,an identifier of an adjacent shelf, recognition of the area the shelf islocated therein, or the like.

In some exemplary embodiments, the device may be configured to applycomputerized learning to improve the identification of items based onfeatures that can be learned from sensors, such as shape, weight,temperature, or the like, or the type of the action, such as based onthe movement pattern, acceleration, or the like. The information may beverified using the scanner or camera (such as inside the stores) toaccurately identify the item.

In some exemplary embodiments, the device may comprise one or moreanti-tampering measures so that it is possible to know whether acustomer is removing (such as intently or non-intently) during shopping.In view of such determination, it may be assessed whether the customeris trying to steal, whether the content of the shopping cart iscorrectly captured by the corresponding virtual cart, or the like. Insome exemplary embodiments, using input from the sensors of the device,the store system may identify that the customer removed the device andinserted an item to the cart. In some exemplary embodiments, theanti-tampering measures may be hardware or software measures implementedin the device and configured to detect an attempt to remove the wearabledevice, to block view of the sensors of the wearable device, to blockcommunication of the wearable device, or the like. In some exemplaryembodiments, a tamper detection system, such as disclosed in U.S. Pat.Nos. 9,629,420, 8,547,222, 8,831,627, 5,959,533, U.S. Patent ApplicationPublication 2011/0,109,461, U.S. Pat. Nos. 7,098,792, 5,650,766,5,298,884 all of which are incorporated by reference in their entiretywithout giving rise to disavowment. In some exemplary embodiments,anti-tampering detection may be performed only after the check-inprocess is performed and until checkout is completed. In some exemplaryembodiments, once the customer leaves the store (e.g., indicated by ageo-fence, identified using a specific beacon, or the like), tamperingdetection may be disabled. The customer may be allowed to remove thedevice. Additionally or alternatively, during such time actions of thecustomer may no longer be tracked and monitored. In some exemplaryembodiments, anti-tampering detection and analysis may be disabledduring bathroom breaks, such as when the customer exits an activeshopping area, but before the customer completed her purchase. Once thecustomer returns to the active shopping area, analysis andanti-tampering may be re-enabled. Additionally or alternatively, asecond check-in process may be performed when the customer re-enters theactive shopping area to ensure that the device is active.

Yet another technical solution is utilizing the wearable smart devicefor determining geospatial location of objects in changing environments.In response to determining based on the visual input of the wearabledevice that the object was picked from one location and placed inanother location, a mapping of items in a facility may be updated toindicate the object is located at the other location. As an example, thewearable device may be utilized by the store staff, such as by theretailer, the workers, or the like, in order to create athree-dimensional mapping of the products in the store. The device maybe worn by the workers when arranging the products in the store. Thethree-dimensional mapping may be utilized for multiple uses, such as foridentifying the items based on their location, for store uses such asstock check, or the like. Additional information may be fed to thedevice by the worker during the arrangement, such as the barcode,information about the item, new items, or the like. Additionally oralternatively, input may be provided via a secondary device, such as aterminal connected to the device, a mobile app coupled to the device, orthe like.

Yet another technical solution is utilizing the wearable smart devicefor removing or deactivating theft detectors. In some exemplaryembodiments, the device may comprise a component that is configured tolegally disable the theft detector component, such as a magnet forlowering the buzzer, deactivation pad, or the like. When the componentis used, the system may identify the product for which the theftdetector is disabled. The identified product may be considered aspurchased, and may be added to the list of products purchased. Afinancial transaction may be performed, enabling a seamlessself-shopping experience, that ensures that items are not shoplifted andpaid for.

Yet another technical solution is utilizing the wearable smart devicefor manual fulfillment of a shopping order of a customer by a picker. Insome exemplary embodiments, the shopping order may comprise a list ofitems selected by the customer, such as in an on-line shopping platform,in a website of the retailer, or the like. A picker that picks items forthe customer may wear the smart wearable device. The visual inputobtained from sensors located on the wearable device to identify whenthe picker picks up an object and places it in a tote associated withthe shopping order of the customer. A corresponding item matching theobject in the list of items may automatically be marked as fulfilled.Additionally or alternatively, in response to identifying a mismatchbetween the object and the list of items, such as that the object is notcomprised by the list of items, the picker may be alerted of themismatch. Additionally or alternatively, the picker may perform multiplepicking tasks for multiple customers simultaneously. The picker mayutilize multiple wearable devices, each of which associated with adifferent customer and paired to a respective tote within the cart.Additionally or alternatively, the picker may utilize a single wearabledevice. In some exemplary embodiments, it may be identified into whichtote the item was placed, and accordingly update the correspondingcustomer order, so that the same picker may pick items for multiplecustomers simultaneously. It may be noted that the wearable device forthe manual fulfillment of a shopping order of a customer by a picker,may be configured to be worn on other organs of the picker, such as onthe chest, collar, forehead or otherwise mounted on the head of thepicker, or the like, in a manner enabling capturing actions of the handsof the picker. Such embodiments may provide a wider scene capturing thehand actions and the objects, being more comfortable for the picker whenfulfilling multiple orders simultaneously, or the like. In suchembodiments, concerns such as violating privacy of the picker,preventing identification of the customer, or the like, may not beinfluential.

Yet another technical solution is utilizing the wearable smart devicefor safety verification. The hand action identified based on the visualinput may be compared with a safety rule to determine conformation orviolation thereof. In response to a violation of the safety rule, asafety alert may be issued to the subject, to a supervisor, or the like.As an example, the safety rule may be a rule relating to anadministration of medicine to a patient, such as a type of the medicine,a dosage, a procedure of administration, prevention of mixture withother medications, allergies or sensitivity for drugs of the patient, orthe like. As another example, the safety rule may be a rule relating tooperating a machine, a vehicle or the like, such as rules related tooperating an airplane, a rule associated with pressing each button, orthe like. Additionally or alternatively, the wearable smart device maybe utilized for other verification process, such as monitoring a seriesof actions of the hand compared to predetermined rules, order of theactions, or the like. As an example, the rule may be related to aninstallation of a product, based on a series of actions on differentcomponents that should be performed in a certain order to provide afunctional product. Yet another technical solution is utilizing thewearable smart device for monitoring a health-care system. In someexemplary embodiments, a health-care worker may be required to wear thewearable smart device. The wearable device may be configured tocontinuously monitor the hand of the health-care worker during treatmentof patients. Each determined action may be automatically reported alongwith the patient information, results or objects associated with theaction, or the like. As an example, in response to a routine set ofactions performed by a nurse, a set of measurements, such as a bloodpressure, body temperature, or the like, may be automatically registeredin the patient medical report, along with the time of performing theaction, identifier of the nurse, or the like. As another example, theaction may be administrating a medicine to a patient. The responsiveaction may be reporting the administration, along with propertiesthereof, such as type, dosage, or the like, to enable monitoringmedication administration for the patient, side effects, or the like.Additionally or alternatively, the responsive action may be preventingviolation of safety rules. As an example, the health-care worker may beadministrating a wrong medicine, a wrong dose, a wrong medicaloperation, or the like, on the patient. The responsive action may be toalert the health-care worker of such mistake. Additionally oralternatively, the responsive action may be to alert another health-careworker to assist the health-care worker. Additionally or alternatively,the responsive action may automatically prevent the administration ofthe wrong medical operation or medicine, such as blocking an intravenous(IV), preventing the medicine to reach the patient, halting medicaldevices utilized for performing the operation, or the like.

One technical effect of utilizing the disclosed subject matter isenabling an efficient hand action tracking without violating privacy ofmonitored subjects and surrounding subjects. The disclosed subjectmatter may enable monitoring mainly the interior portion of the hand,while a wider scene may be blocked by the hand. Personal identificationfeatures, such as in the face, name tags, or the like, may not becaptured by the utilized sensors. Furthermore, the data obtained bysensors utilized in the disclosed subject matter may be limited andfocused only on information essential for determining the action and theobject the action being performed on. The disclosed subject matter mayspare tracking and monitoring the entire environment (such as the entirestore, the entire hospital room, the entire lab, or the like), and thusreducing costs, not requiring changes in the design or additionalequipment, not revealing sensitive data in the environment, or the like.

Another technical effect of utilizing the disclosed subject matter isproviding for a reliable self-checkout shopping service, with anenhanced user experience for customers in retail stores. Utilizing thedisclosed subject matter enables the customer to perform a fast andefficient self-service shopping, while reducing the time that theconsumer spends at the store by avoiding waiting in line, avoid scanningof items, reducing billing time, or the like. Furthermore, the disclosedsubject matter provides economic benefits to the customers, as thedisclosed subject matter enables maintaining competitiveness in themarket, reduce labor costs and increase profitability of retailers whichmay lead to lowering prices, or the like. Additionally or alternatively,the disclosed subject matter provides a seamless shopping experience,lacking a feeling of actual payment. The disclosed subject matterprovides such benefits while preserving the privacy of consumers, withno intrusive tracking of the consumer, without capturing face orpersonal images, or the like.

Yet another technical effect of utilizing the disclosed subject matteris providing for a healthier self-shopping experience. Utilizing thedisclosed subject matter enables avoiding the health risk associatedwith waiting in lines for check-out points, minimizing physicalinteraction with store workers, only the customer touches the itemswhile shopping, the customer uses her shopping bag, no need tophysically pay using cashier machines or passing money or card to otherpeople, or the like. By providing such benefits, the disclosed subjectmatter contributes to prevention of spreading infectious diseases, lowerviral contagion, and the like.

Yet another technical effect of utilizing the disclosed subject matteris providing for a reliable self-checkout shopping service, with anaffordable price to the retailer side. By enabling an improved consumershopping experience, the disclosed subject matter may improve consumerattraction and satisfaction, leading to more consumers hiding theretailer store. By reducing the purchase time for each consumer,additional consumers can be served. Furthermore, the disclosed subjectmatter enables redeploying staff personnel to enhance direct customerservice, maximum floor space, saving labor costs, or the like.

Yet another technical effect of utilizing the disclosed subject matterfor the retailer side, is enabling the service in crowded stores withoutlimiting the number of shoppers that can simultaneously shop in thestore. While regular cashier-free stores, such as stores providingself-service shopping based on in-store monitoring equipment, mayoperate with a limited number of customers inside the store, such asbecause one customer may block the view of other customers, block theview of the monitoring equipment, or the like; the disclosed subjectmatter may enable self-service shopping also in crowded store, such asstores with a maximal number of customers that can fit in.

Yet another technical effect of utilizing the disclosed subject matteris providing for a manner of collecting data by the consumers, that maybe utilized for advertising and enhancing business plans, withoutbothering the consumers or harming their privacy. The data collected bythe disclosed subject matter may be utilized to improve store operationsand checkout experience, to merge consumers online and offlineidentities, to re-target shoppers online based on in-store purchases, tolearn shopper interests and habits, to learn consumers' reaction tovarious shelf displays and store layouts, or the like. Additionally oralternatively, the data collected by the disclosed subject matter may beutilized to extract data useful for retailer operation. As an exampleTRAX™ is a system the employs computer vision technology, artificialintelligence, fine-grained image recognition, and machine learningengines to convert store images into shelf insights. As an example,TRAX™ is able to recognize products that are similar or identical, suchas branded drinks or shampoo bottles but can also differentiate betweenthem based on variety and size. One effect of the disclosed subjectmatter may be to collect visible data that can be used to be analyzed byTRAX™ or other similar products, without the need to send dedicatedpersonal or sensors. Instead, the data is collected in a crowd-sourcingmethodology and as a side-effect to the customers' and pickers' regularactivities.

Yet another technical effect of utilizing the disclosed subject matteris detecting and preventing shoplifting events during shopping. Byutilizing the disclosed subject matter, every item picked by theconsumer may be automatically calculated to be paid for. The consumermay be devoid of controlling or deactivating the monitoring during theshopping session (e.g., no button, a disabled button, or the like), anyattempt to remove the wearable device or hide view of sensors thereofmay be detected and alerted, or the like.

Yet another technical effect of utilizing the disclosed subject matteris automatically generating an accurate store-wide mapping of items andtheir locations. In some cases, the mapping may be generatedautomatically and iteratively according to actions of clerks, pickers,shoppers, or the like, and reflect in real-time and accurately thelocation of each item in the store.

Yet another technical effect of utilizing the disclosed subject matteris to automatically provide for a safety system that monitors humanactivity and identifies violations of safety rules. Such system mayachieve a reduction in accidents and incidents caused due to humanerror.

The disclosed subject matter may provide for one or more technicalimprovements over any pre-existing technique and any technique that haspreviously become routine or conventional in the art. Additionaltechnical problems, solutions and effects may be apparent to a person ofordinary skill in the art in view of the present disclosure.

Referring now to FIG. 1A showing a schematic illustration of a handaction monitoring wearable device, in accordance with some exemplaryembodiments of the disclosed subject matter.

In some exemplary embodiments, Wearable Device 110 may be a wearabledigital product that may be worn on hand's wrist, such as a smart watch,smart wristband, or the like. Wearable Device 110 may be adapted in sizeand shape to fit to a human hand or wrist. Wearable Device 110 may beworn on the left hand, on the right hand, on a single hand, on bothhands (e.g., comprising two wristbands, one on each hand), or the like.Additionally or alternatively, Wearable Device 110 may be embedded inanother wearable device of the user, such as a smart watch, a bracelet,or the like.

In some exemplary embodiments, Wearable Device 110 may comprise one ormore Sensors 111 located thereon. Sensors 111 may be embedded inWearable Device 110, attachable thereto, or the like. Sensors 111 may beconfigured to cover a full range of view around the hand, such as 360° .Additionally or alternatively, Sensors 111 may be positioned at alocation enabling capturing a view of the inner portion of the hand, thepalm of the hand, the base portion of the fingers, or the like, suchthat when a subject is holding an object, Sensors 111 may capture theobject being held, at least partially. Additionally or alternatively,Sensors 111 may be located at the base of the palm, at the wrist, or thelike. Sensors 111 may be in a predetermined constant distance from eachother, may overlap, or the like. Additionally or alternatively, Sensors111 may be dispersed non-uniformly over Wearable Device 110, in order toprovide the full range of view, provide a view enabling identificationof actions and items, or the like. As an example, more sensors ofSensors 111 may be located in the portion of Wearable Device 110 that isconfigured to face the interior portion of the hand. In some exemplaryembodiments, Sensors 111 may comprise visual sensors such multiplecamera lenses, different cameras, LiDAR scanners, ultrasonictransductors, RF-based sensors, other sensors or components havingalternative or equivalent technology, a combination thereof, or thelike. Sensors 111 may be configured to capture pictures, videos orsignals around Wearable Device 110. Other types of input may beprovided, such as heat maps, thermal images, or the like. Additionallyor alternatively, Sensors 111 may comprise motion sensors or detectorsconfigured to recognize any movement of Wearable Device 110 and supporttracking disposition of an item. Additionally or alternatively, Sensors111 may comprise a barcode scanner configured to support identificationof an item.

In some exemplary embodiments, Wearable Device 110 may be utilized torecognize that the subject wearing Wearable Device 110 is about toperform an action (or is performing the action) on an item, to identifythe item being held, information thereabout, or the like. WearableDevice 110 may be utilized to track actions of the hand being wornthereon, items the hand performs or avoids performing the actionthereon, or the like. Sensors 111 may be configured to recognize whenthe hand is approaching an object, picking, holding (e.g., the objectstays constant at the hand), moving the object (e.g., background picturechanged), releasing the object, or the like. Additionally oralternatively, Sensors 111 may be configured to identify parameters ofthe item or enable identification thereof, such as type, category, name,shape, size, price, or the like. In some exemplary embodiments, WearableDevice 110 may be configured to identify a hand-based action that is notintended as a command to the device itself (e.g., a gesture intended asa purposeful command). As an example, Wearable Device 110 may beutilized to identify a picking up action performed naturally, as opposedto a purposeful gesture with the hand that may be performed specificallywith the intent to instruct the device. In some exemplary embodiments,Wearable Device 110 may be configured to identify actions that areperformed as part of the regular interaction of the subject with theitems, and no dedicated actions or gestures by the subject are reliedupon.

In some exemplary embodiments, Sensors 111 may comprise one or moremotion sensors or detectors. Input from the motion sensors may beutilized to support tracking disposition of items upon which the handsperform actions. The motion sensors may be configured to recognize anymovement of Wearable Device 110.

Additionally or alternatively, Sensors 111 may comprise a barcodescanner. Barcode scanner may be utilized to scan barcodes associatedwith items to support identification thereof, provide additionalinformation, such as price, weight, or the like.

Additionally or alternatively, Wearable Device 110 may comprise anAnalysis Component 113, such as a chip or another hardware technology,configured to receive, collet and process pictures, videos signals, orthe like captured by Sensors 111. Additionally or alternatively,Analysis Component 113 may be a transmitter utilized for transmittinginput captured by Sensors 111 to a backend device configured to performthe respected analysis. Such transmitter may be configured to utilize awireless connection, such as Wi-Fi network, Bluetooth, RF transmission ,IR transmission, or the like, for transmitting the data. It may be notedthat all functionalities of Wearable Device 110 (and similarly 120, 140,160 and 170) may be based on on-device computations or on off-devicecomputations, such as performed by an edge device, a remote server, acloud-based server, or the like.

Additionally or alternatively, Wearable Device 110 may be configured toprovide an output to the subject or other user, such as for informingthat the item is identified. As an example, a green light may belightened as positive signal. Other types of signals, such as audiosignals, vibration signals, or the like, may be used. Such signals maybe provided using an Output Component 112, such as a small screen, amicrophone, a Light Emitting Diode (LED), or the like. In some exemplaryembodiments, Output Component 112 may be configured to provide output tothe subject (e.g., LED lighting up in green) indicating of an update ofher virtual cart, such as in view of an addition of an item thereto.Additionally or alternatively, Output Component 112 may be configured toprovide output to the subject (e.g., LED lighting up in red) indicatingof an invalidating of her virtual cart, such as in view of anmisidentification of an item in the shopping cart, identification of atampering event, placing an item in a wrong shopping cart, or the like.

In some exemplary embodiments, Wearable Device 110 may comprise a BuzzerDeactivator 114. Buzzer Deactivator 114 may be configured to enableactivating and deactivating of a theft detector associated with items,such as in stores, pharmacies, or the like. As an example, an employeein a shop may utilize Buzzer Deactivator 114 for activation of the theftdetector and embedding of item's details, such as: item description, barcode, price, or the like, besides to physically attaching the theftdetector to the item. As another example, a shopper may utilize BuzzerDeactivator 114 for deactivating the theft detector physically attachedto the item, physically remove it from the item (if applicable), or thelike.

In some exemplary embodiments, each of the wearable devices depicted inFIGS. 1A-1D, such as Wearable Device 110 may be utilized as a retailsmart device. Wearable Device 110 may be configured to be worn by ashopper during self-service shopping, may be configured to be worn by apicker fulfilling an online order, by a retailer or an employee placingstock, or the like. As another example, the device may be worn by acashier during checkout activity, such as to scan the products andcreate the digital shopping list. Additionally or alternatively,Wearable Device 110 may be utilized for other tasks, such as safetymonitoring, augmented reality games, virtual reality applications, orthe like.

Referring now to FIG. 1B showing a schematic illustration of a handaction monitoring wearable device, in accordance with some exemplaryembodiments of the disclosed subject matter.

In some exemplary embodiments, Wearable Device 120 may be anotherembodiment of the hand action monitoring wearable device. Similar toWearable Device 110, Wearable Device 120 may be adapted in size andshape to fit to a human hand or wrist, may be worn on the left hand, onthe right hand, on a single hand, on both hands may be embedded inanother wearable device of the user, or the like. As an example,Wearable Device 120 may be worn on the wrist of Hand 130.

In some exemplary embodiments, Wearable Device 120 may be configured tocontinuously monitor Hand 130 between a check-in and check-outactivities. Such monitoring may comprise obtaining and analyzing inputrelated to Hand 130, such as visual input, geospatial location, or thelike.

In some exemplary embodiments, Wearable Device 120 may comprise a VisualSensor 122 configured to continuously capture Hand 130. Visual Sensor122 may be configured to capture at least an Interior Portion 132 ofHand 130. Interior Portion 132 may comprise Distal Portion 134 of a Palm133. Visual Sensor 122 may be configured to face Palm 133 wherebycapturing Distal Portion 134. The visual input may capture at least aportion of the object when the object is being held by Hand 130, such aswhen being grasped by Fingers 136 of Hand 130, or the like. At least aportion of the visual input of Visual Sensor 122, such as about 5%,about 10%, about 50%, may comprise a view of Interior Portion 133 toenable identification of the object.

It may be noted that a view of Visual Sensor 122 may be blocked, atleast in part, by Hand 130. As a result, Visual Sensor 122 may not beenabled to capture the whole environment surrounding Hand 130, such asthe face of the user, other people in the surrounding environment,unrelated objects, or the like. Additionally or alternatively, the viewof Visual Sensor 122 may be a spherical view capturing 360 degreepanoramic space surrounding Hand 130. In some exemplary embodiments, thespherical view may have a relatively limited view, such as a sphericalview with a radius of up to about 10 centimeters around Hand 130, up toabout 25 centimeters around Hand 130, or the like.

In some exemplary embodiments, Visual Sensor 122 may be positioned on aprotrusion of Wearable Device 120, distancing Visual Sensor 122 from thesurface of Hand 130. Such placement may be useful for preventing theview to be blocked by the base of the Palm 133.

In some exemplary embodiments, Wearable Device 120 may be configured toprovide images captured by Visual Sensor 122 to be utilized to anAnalysis Module 124. Analysis Module 124 may be configured to identifyan action performed by Hand 130 and to identify an object upon which theaction is performed. Analysis Module 124 may be embedded within WearableDevice 120 or may be located on a device external thereto, such as on aserver, a backend device, or the like.

In some exemplary embodiments, Wearable Device 120 may comprise aCommunication Unit 126 configured to connect Wearable Device 120 to acontroller (not shown) external to Wearable Device 120. The controllermay be configured to determine a responsive action based on the actionor the item. The responsive action may be associated with the purpose ofmonitoring actions of Hand 130, such as reporting the action or theobject, calculating a check based on the action and the object, issuingan alert based on the action or the object, or the like.

In some exemplary embodiments, Wearable Device 120 may be devoid of ade-activation interface for the user. Activation and de-activation ofWearable Device 120 may be performed automatically by the controller. Insome exemplary embodiments, power source (not shown) of Wearable Device120, such as battery, may be sealed and the subject may not have accessthereto. Additionally or alternatively, Wearable Device 120 may beprovided with a limited de-activation interface for the user, thatenables the user to de-activate Wearable Device 120 upon finishing ashopping session, based on permission from the controller, or the like.

In some exemplary embodiments, Wearable Device 120 may be configured tobe utilized for self-service shopping. Wearable Device 120 may beconfigured to be utilized to identify items grabbed by Hand 130 andmoved to or from a physical shopping tote of the user, wherein the itemsare identifiable based on input of Visual Sensor 122. Wearable Device120 may be configured to be associated with a virtual cart uponinitiating a self-shopping session. The virtual cart may indicate a listof items shopped by the user. The virtual cart may be automaticallyupdated based on items moved to and from the shopping cart by Hand 130.In some exemplary embodiments, Wearable Device 120 may comprise aTampering Detection Module 128. Tampering Detection Module 128 may beconfigured to monitor and detect a tamper event during a shoppingsession of the user. Tampering Detection Module 128 may be configured toavoid monitoring user activity outside the shopping session.

Additionally or alternatively, Wearable Device 120 may be configured tobe utilized for manual fulfillment of a shopping order of a customer.The shopping order may comprise a list of items. Hand 130 may be of apicker tasked with picking items to fulfill the shopping order of thecustomer. Wearable Device 120 may be configured to identify actions ofpicking up an object by Hand 130 and placing the object in a toteassociated with the shopping order of the customer.

Additionally or alternatively, Wearable Device 120 may be configured tobe utilized for protecting the user or other related subjects. Theresponsive action determined based on the input of Visual Sensor 122 maycomprise comparing the action performed by Hand 130 with a safety rule.In response to a violation of the safety rule, a safety alert may beissued. Additionally or alternatively, Wearable Device 120 may beconfigured to be utilized for monitoring a health-care system. WearableDevice 120 may be configured to continuously monitor the hand ofhealth-care workers during treatment of patients.

Referring now to FIG. 1C showing a schematic illustration of a handaction monitoring wearable device, in accordance with some exemplaryembodiments of the disclosed subject matter.

In some exemplary embodiments, Wearable Device 140 may be anotherembodiment of the hand action monitoring wearable device. WearableDevice 140 may comprise a First Wearable Component 142 and a SecondWearable Component 144. First Wearable Component 142 and Second WearableComponent 144 may be connected in a manner allowing detaching one fromthe other, re-connecting therebetween, or the like. As an example, thetwo components may be connected using magnetic force, using a physicalconnector, or the like. Additionally or alternatively, the twocomponents may be placed adjacently to one another but remain decoupled.First Wearable Component 142 may be configured to be worn on a FirstHand 152. Second Wearable Component 144 may be configured to be worn ona Second Hand 154. First Hand 152 and Second Hand 154 may belong to asingle user such as a left hand (142) and a right hand (144), to twodifferent related users, such as two users shopping together, or thelike. Each of Wearable Component 142 and Second Wearable Component 144may be configured to monitor First Hand 152 and Second Hand 154,respectively, as independent wearable device. However, the analysis ofvisual input provided thereby to determine the action and object, may beperformed uniformly on a combination of visual inputs of each component.

In some exemplary embodiments, First Wearable Component 142 may comprisea First Sensor 143 (or multiple sensors as in Wearable Device 110).Second Wearable Component 144 may comprise a Second Sensor 145 (ormultiple sensors as in Wearable Device 110). The visual input utilizedto determine the actions and the associated object may comprise bothinput from First Sensor 143 and from Second Sensor 145. Additionally oralternatively, First Wearable Component 142 and Second WearableComponent 144 may be synchronized, the visual input provided by FirstSensor 143 and from Second Sensor 145 may be combined and provided byone of the two components, or the like. Additionally, or alternatively,the analysis of the two visual inputs may be inter-connected. Forexample, in view of products and locations identified based on onedevice, the computations associated with the analysis of the input fromthe second device may be modified. As an example, using the view fromone device, a potential location of the user may be determined, and theset of potential items that can be captured in the second device may bedetermined based on the location, and utilized when analyzing the inputfrom the second device.

Additionally or alternatively, both First Wearable Component 142 andSecond Wearable Component 144 may be worn together on a single Hand 150.In such a case, Wearable Component 140 may be configured to act similarto Wearable Device 120 (or Wearable Device 110, in case of comprisingmultiple sensors).

Referring now to FIG. 1D showing schematic illustrations of a handaction monitoring wearable device, in accordance with some exemplaryembodiments of the disclosed subject matter.

In some exemplary embodiments, Wearable Device 160 and Wearable Device170 may be embodiments of the hand action monitoring wearable device.Wearable Device 160 may be of a shape of glove configured to be worn ona hand of a user. Wearable Device 160 may comprise one or more Sensors162 located on the inner side of one or more of the fingers.

In some exemplary embodiments, Sensors 162 may comprise visual sensors(such as in Sensors 111 and Visual Sensor 122) configured to capture atleast an interior portion of the hand, such as the palm, the portionbetween the palm and the fingers, the portion between the palm and thewrist, or the like. Sensors 162 may be configured to capture at least aportion of the object when the object is being held by the hand.

In some exemplary embodiments, Wearable Device 160 may be configured toprovide visual input (or other types of input) captured by Sensors 162.An action performed by the hand an object upon which the action isperformed may be determined based on the visual input (or the othertypes of input). In some exemplary embodiments, Wearable Device 160 maycomprise a communication unit (not shown) configured to connect WearableDevice 160 to a controller (not shown) external to Wearable Device 160.The controller may be configured to determine the action, the item, aresponsive action based on the action or the item, or the like.

In some exemplary embodiments, Wearable Device 160 may comprise othersensors located on other portions of the hand, such as a thermometer, anaccelerometer, a gyroscope, a weight sensor, or the like.

Additionally or alternatively, Wearable Device 170 may be of a shape ofone or fingers cap configured to be worn on one or more fingers of ahand of a user. Wearable Device 170 may comprise one or more Sensors 172located on the inner side of one or more of the fingers which uponWearable Device 170 is worn. Sensors 172 may comprise visual sensors(such as in Sensors 111, Visual Sensor 122, Sensors 162) configured tocapture at least an interior portion of the hand, such as the palm, theportion between the palm and the fingers, the portion between the palmand the wrist, or the like. Sensors 172 may be configured to capture atleast a portion of the object when the object is being held by the hand.

In some exemplary embodiments, other embodiments of the wearable devicemay be utilized, such as a device configured to be worn on the chest ofthe user, embedded in a vest to be worn by the user, a hat shaped deviceconfigured to be worn on the head of the user, a device configured to beworn on the forehead of the user such as using elasticized straps, orthe like. Such wearable devices may also comprise visual sensors (suchas in Sensors 111 and Visual Sensor 122) configured to capture at leastan interior portion of the hand of the user, objects being held by thehand user actions performed by the hands of the user, or the like.

Referring now to FIG. 2 showing a schematic illustration of an exemplaryself-service shopping scheme, in accordance with some exemplaryembodiments of the disclosed subject matter.

In some exemplary embodiments, a Retail Store 200 may provide aself-service shopping, using smart wearable devices, such as WearableDevice 110, 120, 140, 160, and 170, or the like. It is noted that allfunctionalities of the smart wearable devices may be based on on-devicecomputations or on off-device computations, such as performed by an edgedevice in Store 200, a remote server, a cloud-based server, or the like.

In some exemplary embodiments, when a Customer 210 enters Store 200, orchecks into a system of Store 200 (such as by using a Mobile Device 212or otherwise), Customer 210 may receive one or more retail smartwristbands, such as Wearable Device 214, from Store 200. In someexemplary embodiments, Store 200 may have a designated location (205)where wearable devices are placed and await to be picked up bycustomers, similarly to the location of the Available Carts 206. In someexemplary embodiments, in Location 205, the wearable devices may becharged, to ensure they have sufficient power level. Wearable Device 214may be affiliated or assigned to Customer 210 for the duration of theshopping session. Customer 210 may be requested to pair or synchronizeWearable Device 214 to a device thereof, such as through a designatedmobile app of Mobile Device 212, by registering to an account associatedwith Store 200, by scanning Wearable Device 214 using a Scanner 201, orthe like. After the relevant pairing, each action performed with thehand(s) of Customer 210 wearing Wearable Device 214 may be attributed tothe account of Customer 210.

Additionally or alternatively, the shopper, such as Customer 220, mayutilize his personal Wearable Device 222 to perform self-serviceshopping in Store 200. When Customer 220 enters Store 200 while wearingWearable Device 222, Wearable Device 222 may be configured to recognizethat Customer 220 has entered Store 200, and may synchronize with thestore's system. As an example, Wearable Device 222 may connect to theWi-Fi of Store 200, and accordingly recognize Store 200. As anotherexample, Wearable Device 222 may be preconfigured to recognize Store 200and connect to systems thereof. As yet another example, Wearable Device222 may be manually paired with systems of Store 200, such as byscanning an identifier thereof using Scanner 201, pairing with ashopping cart from Available Carts 206, or the like. As yet anotherexample, Wearable Device 222 may be paired to a mobile device ofCustomer 220, which may be utilized to manually check in Store 200, mayconnect via a respective mobile app, or the like. Additionally oralternatively, a geo-fence associated with Store 200 may be used todetect entering and exiting Store 200. Wearable Device 222 may beconfigured with geo-fence or location-based activation. Wearable Device222 may be activated when Customer 220 enters Store 200, by usinggeo-fence feature or other signal to a mobile app, or directly toWearable Device 222, which may activate Wearable Device 222. It may benoted that as Wearable Device 222 is the shopper's property (e.g., notreceived in the course of check-in to Store 200), the process of checkin may activate or turn on Wearable Device 222, and then each actionperformed with the hands wearing Wearable Device 222 within the storewill be attributed to Customer 200. Furthermore, Customer 220 mayutilize a personal Shopping Bag 224 instead of a shopping cart of Store200, for environmental purposes, for convenience, in order to performfaster shopping, or the like.

In some exemplary embodiments, upon entering Store 200, a physicalbarrier (not shown) may prevent customers from entering the shoppingarea before performing a check-in activity. As an example, a carouselbarrier may prevent the customer from entering until the customercompletes a pairing of her wearable device. After the check-in activityis completed, monitoring of the activity of the customer may commence.

In some exemplary embodiments, each wearable device may be utilized tocontinuously monitor actions of the shopper wearing the wearable deviceduring the shopping session. The monitoring may be performedcontinuously between the check-in activity and a respective checkoutactivity. In some exemplary embodiments, Wearable Device 222 may beconfigured to provide information to the owner of Store 200 only withrespect to the shopping session within Store 200. As an example, ifCustomer 220 uses Wearable Device 222 in several stores, each store maygain an access to information relating to visiting the respective storeonly and not to information relating to other stores. In some exemplaryembodiments, analytics and general information may be tracked by ageneral service provider, who may not necessarily be affiliated to anyspecific store.

In some exemplary embodiments, a Customer 240 may be wearing a WearableDevice 242 during shopping in Store 200. Sensors of Wearable Device 242may be configured to capture at least an interior portion of the hand ofCustomer 240 wearing Wearable Device 242, and provide visual inputthereof. Wearable Device 242 may be configured to identify each itemthat Customer 240 performs an action with (such as picking, holding,putting in a shopping bag or cart, or the like), based on the visualinput. As an example, Wearable Device 242 (or an associated softwarethereof, on-device or in a back-end) may be configured to identify thatCustomer 240 picks up Object 245, and place it in Shopping Bag 246.Additionally or alternatively, Wearable Device 242 may be configured toutilize additional types of input to identify the object and the action,such as positioning readings of Wearable Device 242, accelerometerreadings of Wearable Device 242, or the like. In response to detectingthe object and the action performed thereon by Customer 240, WearableDevice 242 (or the associated software thereof) may be configured toupdate a virtual cart of Customer 240 to include Object 245.Additionally or alternatively, Wearable Device 242 may be configured todetermine, such as based on a series of visual inputs over time, thatCustomer 240 has decided not to purchase Object 245. As an example,determining that customer 240 removed Object 245 from Shopping Bag 246,placed Object 245 back to its location, to another location in Store200, or the like. In response to such determination, Wearable Device 242(or the associated software thereof) may be configured to update thevirtual cart of Customer 240 to exclude Object 245. In some exemplaryembodiments, the virtual cart may be displayed for Customer 240, such asusing Mobile Device 244, or the like. Additionally or alternatively, thevirtual cart may be retained by systems of Store 200 and sent toCustomer 240 upon finishing the shopping session. Additionally oralternatively, Wearable Device 242 may be configured to emit an auditorycue such as a beep, or a visual output such as a green light, or thelike, indicating the addition (or removal) of an object to the virtualcart.

In some exemplary embodiments, some actions or objects may beunidentified or recognized. In such case the system output may“disqualify” or invalidate the entire virtual cart. As an example: theanalysis software that is configured to analyze the visual input anddetermine the action, may not be able to analyze certain actions suchas: picking an item from the floor, moving an item from one cart toanother, which will cause such disqualification, or the like.Additionally or alternatively, if the visual sensor captures a view ofthe content of Shopping Bag 246 and an unknown item is identifiedtherein, the virtual cart may be disqualified.

Additionally or alternatively, the smart wearable device may beconfigured to identify removal or tampering thereof during the shopping.As an example, Wearable Device 242 may be configured to detect anytampering event aimed at interfering with monitoring of the hand ofCustomer 240. Wearable Device 242 may be configured to detect removal ofWearable Device 242, blocking view of the sensors embedded in WearableDevice 242, hiding an object with another object, or the like. WearableDevice 242 may utilize input readings from other sensors thereof,besides the visual sensors, such as readings of an accelerometerthereof, to determine a behavioral pattern of Customer 240 that isindicative of an attempt to prevent monitoring of the hand of Customer240. Additionally or alternatively, Wearable Device 242 may comprise adetector or sensor that is configured to identify an attempt or actionfor tampering or removing Wearable Device 242. Additionally oralternatively, Wearable Device 242 may be configured to emit aheart-beat signal to indicate that it is active. In case the heart-beatsignal is not received for a predetermined time window, such as oneminute, 10 seconds, or the like, it may be determined that the devicewas tampered with, deactivated, depleted its power source, or the like.In some exemplary embodiments, such event may be considered a tamperingevent. In response to detecting a tampering event, Wearable Device 242may be configured to perform an anti-tampering action, such as issuingan alert, indicating a staff member of Store 200, or the like. It may benoted that Wearable Device 242 may be configured to perform suchdetection only during the self-service shopping session of Customer 240.After activation (e.g., check-in activity) of Wearable Device 242, anyattempt to remove or actual tamper or removal of Wearable Device 242 maybe signaled to the system, and may cause the shopping bag or cart to bedisqualified. The anti-tampering feature may automatically be turned tooperating mode when the Wearable Device 242 is activated or paired andmay automatically be turned off or enter a sleeping mode when WearableDevice 242 is deactivated (e.g., check-out activity). Customer 240 maybe enabled to perform a tampering event after the self-service shoppingsession ends without resulting in the anti-tampering action.

Additionally or alternatively, the visual input of Wearable Device 242may be supported with visual input capturing the content of Shopping Bag246. Such input may be also captured by Wearable Device 242, such aswhen placing objects in Shopping Bag 246, when being positioned aboveShopping Bag 246, or the like. Identification of objects purchased byCustomer 240 may be supported and validated based on the input capturingShopping Bag 246. In response to a discrepancy between the content ofthe virtual cart and content of Shopping Bag 246, the virtual cart maybe invalidated. Additionally or alternatively, the virtual cart may beupdated accordingly. Additionally or alternatively, the virtual shoppingcart may be validated based on input from sensors located on or embeddedin Shopping Bag 246, such as weighting sensors, cameras, or the like.Additionally or alternatively, the visual input of Wearable Device 242may be utilized to recognize when an item is accidently put in a wrongcart, such as by identifying that the item has been placed in a shoppingcart different than Shopping Bag 246 associated with the customer, bydetermining that the location in which the item is placed is differentthan Shopping Bag 246, or the like. In some exemplary embodiments,Customer 240 may be notified of the misplacement of the item, may berequested to re-place the item in Shopping Bag 246, or the like.Additionally or alternatively, an additional shopping bag may bedetermined to be associated with Customer 240 in addition to ShoppingBag 246. In some exemplary embodiments, when Customer 240 leaves Store200, he will be invoiced for the items which were picked up when thehand wearing Wearable Device 242, and placed in Shopping Cart 246,without scanning the items. The payment may be made using a paymentsolution integrated with or embedded in a mobile app on a Mobile Device244 of Customer 240 and supported by backend system of Store 200. Inresponse to identifying a check-out activity of the subject, performinga transaction based on content of the virtual cart of the subject.Additionally or alternatively, Shopping Cart 246 may be inspected by theretailer before Customer 240 leaving Store 200, such as for validationthereof, in tampering suspected cases, or the like.

In some exemplary embodiments, a Picker 230 may utilize Wearable Device232 for manual fulfillment of shopping orders of customers, such ason-line orders. Each customer may provide a list of items to bepurchased. The list of order may be viewed to Picker 230 via a screen,such as on a Computing Device 234 managing such orders. Picker 230 maypick items listed in the list of items to fulfill the shopping order ofthe customer. Wearable Device 232 may be configured to identify whenPicker 230 picks up an object and place it in a tote associated with theshopping order of the customer. In response to identifying acorresponding item to the object in the list of items, suchcorresponding item may be automatically marked as fulfilled.

In some exemplary embodiments, Picker 230 may fulfill multiple orderssimultaneously, one after the other, or the like. Picker 230 may obtainmultiple shopping orders of different customers. The items of eachshopping order may be picked and placed in a tote associated with thecustomer. As an example, Cart 236 may comprise three different totes(237, 238 and 239) each of which utilized to pick a shopping order of adifferent customer. Picker 230 may obtain a combined list of itemssorted according to their location in Store 200 to enable fastercollecting of the items. Each item in the combined list may be marked toindicate the relevant customer. Wearable Device 232 may be configured tomonitor Picker 230 while fulfilling each shopping order. Picker 230 maybe enabled to configure Wearable Device 232 to the relevant customerwhenever switching between the orders. Additionally or alternatively,Wearable Device 232 may be configured to identify to which customer theitem belongs based on identifying the tote in which the item is placed.Wearable Device 232 may be configured to provide visual input capturingthe tote that the item is being placed therein, an identifier thereof,such as a barcode, an identifying color, or the like.

In some exemplary embodiments, two customers may be enabled to shoptogether in one shopping cart and one bill, such as Customer 250 andCustomer 260. Wearable Device 252 worn by Customer 250 and WearableDevice 262 worn by Customer 260, may be synchronized to a singleaccount. In some exemplary embodiments, Wearable Device 252 and WearableDevice 262 may be components divided from a single wearable device, suchas First Wearable Component 142 and Second Wearable Component 144 ofWearable Device 140 depicted in FIG. 1C. Additionally or alternatively,Wearable Device 252 and Wearable Device 262 may be separated wearabledevices that are both paired to the same customer account, virtual cart,or the like. Each object picked by Customer 250 or by Customer 260 maybe added to a single combined virtual cart. In some cases, Customer 250and Customer 260 may utilize a single Shopping Cart 255 to place thepurchased objects, however, multiple shopping carts or bags may beutilized. In some exemplary embodiments, such an embodiment may enable ashopping experience of a family, of a husband and wife, of a parent andhis children, or the like.

In some exemplary embodiments, the wearable device may be utilized forremoval of theft detector. The wearable device may be utilized forself-service purchase in retail stores, where items are protected fromtheft, such as fashion items which are protected with a magnet needle.As an example, Customer 270 may utilize Wearable Device 272 worn therebyduring self-service shopping in Store 200 to remove or disable anAnti-Theft Component 277 from an Item 275. Wearable Device 272 may beconfigured to deactivate, physically disconnect, turn off, or otherwisedisable an anti-theft Component 277 from Item 275. In some exemplaryembodiments, Anti-Theft Component 277 may be attached to Item 275 andpre-configured to the system of Store 200. In some exemplaryembodiments, Anti-Theft Component 277 may be configured to signaldescription of Object 275 to Wearable Device 272. In some exemplaryembodiments, Anti-Theft Component 277 may be activated or deactivated byWearable Device 272. In order to complete a purchase of Item 275 and payfor it, Customer 270 may have to deactivate Anti-Theft Component 277using Wearable Device 272. Such deactivation may be performed beforeCustomer 270 leaves Store 200 or elsewhere, as determined by the store.Customer 270 may be invoiced for Item 275 after deactivating anti-TheftComponent 277. In some exemplary embodiments, the system may wait apredetermined amount of time or until Customer 270 is indicated asleaving Store 200 to send a bill, so as to gather all purchases togetherand send a single bill. Additionally or alternatively, a Worker 280 ofStore 200 may configure the theft detector for the item, by implementingin the theft detector the respective item details, includingdescription, bar code, price, or the like. The theft detector removingcomponent may be embedded in Wearable Device 272, may be a differentwearable item, or the like.

Additionally or alternatively, the wearable device such as WearableDevice 282 may be utilized by the retailer or workers of Store 200, suchas Worker 280, for different purposes, such as arranging Store 200,stocktaking, mapping locations of objects within Store 200, determiningthe exact location of each item in Store 200, inventory checking,verification of the quantities and condition of items in Store 200,mapping shelfs in Store 200, or the like. or the like. Wearable Device282 worn by Worker 280, may be configured to follow each object beingheld by Worker 280 from its position at a Delivery Box 285 to theshelves of Store 200. Visual input from different wearable devicesutilized in Store 200, or other sensors monitoring Store 200, such asCamera 201, may be matched to enable the system to draw the structureStore 200. After the system learns the structure of Store 200, thesystem may be configured to map the location of each item put by Worker280 on the shelves. The system may have a virtual map of the entireStore 200 together with the items put on their shelves or arrangedelsewhere in Store 200. Wearable Device 282 may be configured toidentify an action of placing an Object 284. In response, WearableDevice 282 may be configured to determine a geospatial location ofObject 284 after being placed, and updating a mapping of items in Store200 to indicate that Object 284 is located at the geospatial location.Additionally or alternatively, when a shopper takes an item from ashelve, the wearable device of the shopper may be configured torecognize the shelf (such as when the shopper wrist is close to thatshelf). Such feature may assist with identifying the item, which thesystem has prior knowledge of its location on that shelf, frommonitoring the store arrangement, or the like. Mapping of the shelvesmay be supported by unique identifiers (such as stickers) which may bepasted on, or otherwise affixed to the shelves' fronts. The shelves maybe marked such as using a 2D mapping of aisle number and shelf number inthe aisle. Additionally or alternatively, each shelf may be divided intocells creating a 3D mapping. Each shelf or shelf cell may have a uniqueidentifier (such as combination of letters and numbers) which may easethe process of Wearable Device 282 in mapping Store 200 and furtherrecognition of the exact location of Wearable Device 282 when used byWorker 280, or other wearable devices worn by shoppers. In someexemplary embodiments, the identifier may provide an approximatedlocation. The approximated location may be of size of about 1 meter×1meter×1 meter, or the like. Additionally or alternatively, theapproximated location may be of size of about 80 cm in width, 30 cm indepth and 30 cm in height. The measurements may be based on the size ofthe shelf, such as the height of the shelf, the depth of the shelf, orthe like. In some exemplary embodiments, the approximated location maybe utilized to reduce complexity of identifying the item. For example,based on the approximated location, potential items that are retained inthe approximated location (or nearby locations) may be known and may beused as the “immediate suspects” for matching when the shopper picks upan item. Determining whether the picked up item is a specific item ispotentially an easier computational task than attempting to identifywhich item it is when compared to a database of thousands of items. Insome exemplary embodiments, matching may be performed with respect to afirst repository having a small number of items that are located nearby,and with respect to a second repository having all potential items inthe store. A reduced confidence level may be sufficient for matching anitem in the first repository in comparison to the minimal confidencethreshold required for the second repository. Additionally oralternatively, the matching process may take into account the imagetogether with the distance between the item and the approximatedlocation, such as increasing likelihood of matching when the item isstored nearby the approximated location. In some exemplary embodiments,items that are usually retained in one place may be naturally moved byshoppers to other places that are still nearby.

In some exemplary embodiments, a database or a catalog of items may beretained by the system managing the self-service shopping using thewearable devices. Prior to activating the solution, all items which maybe sold in Store 200 may be pictured from different angels, categorizedand stored in a designated database, e.g., a catalog database. When thesystem (through a wearable device) recognizes that the hand(s) made anaction or got close to an object, the picture, video or signal of theitem may be matched with the database, and identified thereof (or notidentified, if such object is not listed in the database or matching hasnot succeeded). Additionally or alternatively, a partial database may beutilized for each store. The partial database may comprise items thatare known to be in the store in a certain location. In order to speedthe wearable device's identification of an object, instead of searchingthe entire database each time, only the items, which the service isaware of being located in the location of the wearable device, baes onpositioning readings thereof, may be searched. In case that the systemdoes not find the object in that partial database, the system may searchthe whole database for that object. The system may be configured to knowwhat items are located in Store 200 based on an inventory list obtainedfrom different wearable devices worn by customers, from Wearable Device282, from the relevant Point of Sale (PoS) used at the store, othersolution managing the store's inventory, or the like. Each item that islocated in Store 200 may be identified and listed in the partialdatabase of the store.

In some exemplary embodiments, the wearable devices may be configuredprovide an audio guide to items' locations at Store 200. The audio guidemay be utilized for shopping assistance for blind persons or personssuffering from partial blindness or low vision. As an example, thesystem may utilize a shopping list of Customer 240 for that purpose.Wearable Device 242 may be paired Mobile Phone 244 of Customer 240. Anaudio system of Wearable Device 242 may be configured to vocalize theparameters of the object which is identified by Wearable Device 242(such as name, price, or the like). Customer 240 may be able to listento these parameters through the app's voice feature of Wearable Device242, via Mobile Device 244, or the like.

In some exemplary embodiments, the system utilizing the wearable devicesmay be configured to learn shoppers' behavior in general, such as theways and methods for choosing, picking, holding, moving, releasingitems, or the like, the unique way of each shopper to perform theseactions, or the like. Such learning may be performed using machinelearning or other techniques. Learning shoppers' behavior may reducefalse signals or portion of undefined shopper's actions. Additionally oralternatively, the wearable device may be configured to learn propertiesof the items, such as shape, from different angles, and improve theidentification of the items to minimize false or non-identifications.

In some exemplary embodiments, there may be multiple of customers inStore 200. Some of which may utilize wearable devices, and some may not.In some exemplary embodiments, some of the customers may conductself-service shopping while other may conduct traditional shopping whichalso comprise manual scanning of the items, e.g., by a cashier duringcheck-out.

Referring now to FIG. 3 showing schematic illustrations of visual inputsprovided by a hand action monitoring wearable device, in accordance withsome exemplary embodiments of the disclosed subject matter.

In some exemplary embodiments, a wearable device, such as 110, 120 or140 depicted in FIGS. 1A-1C, may be worn on a Hand 310 of a subject. Thewearable device may comprise one or more sensors configured to captureat least an in interior portion of Hand 310. In some exemplaryembodiments, the wearable device may be configured to be worn on a wristof Hand 310, whereby positioning the sensor to face a palm of Hand 310,or at least a distal portion thereof, such that enabling to capture atleast a portion of an object when the object is being held by Hand 310,or when being grasped by fingers of Hand 310, or the like.

In some exemplary embodiments, the sensors may be configured to providevisual input that at least a portion thereof comprises a view of theinterior part of Hand 310, such as Images 301, 302, 303 and 304. As anexample, Image 301 captures a view of Hand 310 in front of a Shelf 350within the store, with portion of the objects on the shelf, such asObject 320. As another example, Image 302 captures a view of Hand 310placing Object 320 in a Basket 340. As yet another example, Image 303captures Hand 310 along with a portion of Basket 340, that comprisesother objects picked by the user, such as Object 355. As yet anotherexample, Image 304 captures a different view of Hand 310 being free fromany object, while approaching other items in the store such as Object360. It may be noted that different images may capture different viewsassociated with Hand 310. However, at least a portion of the image (suchas about 5%, about 10%, about 50%, or the like) may comprise a view ofthe portion of Hand 310. Such portion may vary from one image toanother, based on the angle of Hand 310, the position thereof, theaction being performed thereby, or the like. As an example, Image 301captures a smaller portion of Hand 310 comparing to Image 302.

In some exemplary embodiments, the view of the sensor may be blocked, atleast in part, by Hand 310. As an example, in Image 302 and Image 303,the view of the sensor is limited to a portion of Basket 340, withoutshowing the external environment thereof.

In some exemplary embodiments, the wearable device may be configured torecognize when Hand 310 is getting close to an item (such as Object320), picking, holding (e.g., the object remains being held by Hand310), moving it (e.g., background picture changed), releasing an item,or the like. The identification of Object 320 and the actions performedthereon may be identified based on the visual input, such as Images301-304. As an example, based on Image 301, the wearable device may beconfigured to recognize that Hand 310 is about to perform an action onObject 320, based on identifying that Hand 310 getting close to Object320, and picking it. Based on Image 302, it may be identified that Hand310 is holding Object 320 and moving it towards Basket 340. Based onImage 304, it may be identified that Hand 310 is releasing Object 320and putting it in Basket 340. In Image 304, Hand 310 may be identifiedto be free again and ready to pick another item.

Additionally or alternatively, Images 301-304 may be utilized toidentify the item being held, or about to be held by Hand 310. Imageanalysis techniques may be applied to identify Object 320. Object 320may be recognized based on a catalog of items of the store, such as bycomparing portions of Object 320 with the catalog of items. Additionalinformation may be determined about Object 320, such as the type,category, name, shape, weight, size, or the like, such as by identifyingparameters on Object 320, identifying the location of Object 320, or thelike. Additionally or alternatively, a positioning reading of thewearable device, indicative of the location thereof, may be obtained,such as using a location sensor thereon, a location system of a deviceassociated therewith, or the like. A subset of a catalog of items of thestore may be determined based on the location, such as based on an inputfrom the store, or the like. As an example, the subset of the catalogmay comprise items located on Shelf 350, items located in the fridgecomprising Shelf 350, diary items, or the like. A product recognitionmay be performed to identify Object 320 with respect to the subset ofthe catalog of items.

In some exemplary embodiments, visual input capturing the content ofShopping Bag 340, such as Image 302 and Image 303, may be utilize tovalidate the virtual cart associated with the customer. The content ofShopping Bag 340 may be identified based on the images, and compared tothe virtual cart being updated during the shopping session. In responseto determining a discrepancy between content of the virtual cart andcontent of Shopping Bag 340, the virtual cart may be invalidated,updated based on the images, or the like. As an example, Object 355 maybe identified in Image 303, while not be listed in the virtual cart.Accordingly, the virtual cart may be updated to include Object 355 or anidentifier thereof.

In some exemplary embodiments, the visual input may comprise portions ofthe store in which the shopping session is being performed. As anexample, Image 301 captures a Shelf 350 and Image 304 captures Shelf360. Such images may be analyzed to recognize the shelf, such as basedon an identifier thereof, a sticker pasted thereon, based on a priorknowledge of the location of the associated object on that shelf, frommonitoring the store arrangement, or the like. As an example, based onprior knowledge of the location of Object 320, Shelf 350 may beidentified. Additional action may be performed based on identifying theshelves, such as updating inventory of the respective store to indicatethat Object 320 is purchased, maintaining the mapping of objects in thestore, extracting additional information related to Object 320, such asoffers or sales, expiration date, temperature (based on the type of theshelf, the shelf being in a refrigerator or a freezer, or the like), orthe like.

Referring now to FIG. 4A showing schematic illustrations of exemplaryenvironments in which the disclosed subject matter may be utilized, inaccordance with some exemplary embodiments of the disclosed subjectmatter.

In some exemplary embodiments, the disclosed subject matter may beutilized for monitoring a health-care system, such as in Health-careFacility 400 a. A Wearable Device 412, such as 110, 120, or 140 depictedin FIGS. 1A-1C may be configured to be worn by a healthcare worker, suchas Nurse 410. Wearable Device 412 may be configured to continuouslymonitor actions of Nurse 410 during treatment of patients, such asPatient 415. Wearable Device 412 may be configured to identify an actionperformed by the hands of Nurse 410 during treatment of Patient 415,such as administrating a medicine to Patient 415 using an intravenous.Wearable Device 412 may be configured to identify the object upon whichthe action being performed, e.g., the intravenous tube, the medicationNurse 412 is adding to the intravenous, or the like. Based on the inputprovided by sensors of Wearable Device 412, additional parameters of theaction and the objects may be determined, such as the type of themedication, the medication dose being flushed to intravenous bag, theinitial amount of fluids in intravenous bag before adding themedication, or the like. Other parameters and features may be determinedbased on the previous parameters, such as the rate in which themedication is being flushed through the intravenous tubing, or the like.

In some exemplary embodiments, the action identified to be performed byNurse 410 using Wearable Device 412, may be compared with a safety rulerelating to an administration of medicine to a patient, such asadministering a wrong dosage, administering a wrong drug, performing awrong procedure, administering the medicine to a wrong patient, or thelike. In response to a violation of the safety rule, Wearable Device 412may be configured to issue a safety alert to Nurse 410, or a supervisorthereof. Additionally or alternatively, information determined based onthe input provided by the sensors of Wearable Device 412, may beutilized to automatically update a treatment report of Patient 415, suchas updating the time of providing the medication and associated datasuch as the dose, and side effects, activating a timer to alert whenflushing of the medication using the intravenous is finished and shouldbe replaced, or the like.

Referring now to FIG. 4B showing schematic illustrations of exemplaryenvironments in which the disclosed subject matter may be utilized, inaccordance with some exemplary embodiments of the disclosed subjectmatter.

In some exemplary embodiments, the disclosed subject matter may beutilized for monitoring other types of work spaces, such as in aFacility 400 b. A Wearable Device 422, and a Wearable Device 432, suchas 110, 120, or 140 depicted in FIGS. 1A-1C may be configured to be wornby a Worker 420 and a Worker 430 respectively. Wearable Device 422 maybe configured to continuously monitor actions of Worker 420 duringperforming a task, while Wearable Device 432 may be configured tocontinuously monitor actions of Worker 430. Wearable Device 422 may beconfigured to identify objects being held by the hand of Worker 420, andthe action that the hand performs thereon. Additional parameters of theaction and the object may be determined, such as the type of object, thelocation the object being placed in, a secondary object associated withthe action and the object, or the like. Similarly, Wearable Device 432may be configured to identify objects being held by the hand of Worker430, and the action that the hand performs thereon.

In some exemplary embodiments, the action identified to be performed byWorker 420 or Worker 430, may be compared with a safety rule relating tothe task or work space, such as locating an object in a wrong location,activating a wrong instrument, pushing a wrong button, or the like. Inresponse to a violation of the safety rule, the associated wearabledevice (422 or 423) may be configured to issue a safety alert to Worker420 or Worker 430, or a supervisor thereof. As an example, based on theinput provided by the sensors of Wearable Device 422, Object 452 may bedetermined to mistakenly be left in Car 450 after Worker 420 finishedworking thereon.

Additionally or alternatively, information determined based on the inputprovided by the sensors of Wearable Device 422 and 432, may be utilizedto automatically update a mapping of items in Facility 400 b to indicatethe geospatial location of objects, such as by updating the geospatiallocation of Objects 426 based on the location being placed in by thehands of Worker 420 or Worker 430.

Referring now to FIG. 4C showing schematic illustrations of exemplaryenvironments in which the disclosed subject matter may be utilized, inaccordance with some exemplary embodiments of the disclosed subjectmatter.

In some exemplary embodiments, the disclosed subject matter may beutilized for monitoring other types of work spaces, such as in a Lab 400c. A Wearable Device 442, such as 110, 120, or 140 depicted in FIGS.1A-1C may be configured to be worn by a lab Technician 440. WearableDevice 442 may be configured to continuously monitor actions ofTechnician 440 during performing her job, during performing anexperiment, or the like. Wearable Device 442 may be configured toidentify objects being held by the hand of

Technician 440, such as Object 444, and the action that the handperforms thereon. Additional parameters of the action and the object maybe determined, such as the type of object, the material in the object,the location or instrument that the object or the material being placedin, the equipment utilized to perform the action, a secondary objectassociated with the action and the object, or the like.

In some exemplary embodiments, the action identified to be performed byTechnician 440, may be compared with a safety rule relating to the taskor the associated materials, such as locating an object in a wronglocation, using a wrong equipment, mixing wrong materials, usingdangerous materials, or the like. In response to a violation of thesafety rule, Wearable Device 442 may be configured to issue a safetyalert to Technician 440, or a supervisor thereof, automatically activatea rescuing action, or the like. Additionally or alternatively,information determined based on the input provided by the sensors ofWearable Device 442, may be utilized to automatically updating a mappingof items in Lab 400 c to indicate the geospatial location of objects,such as by updating the geospatial location of each object identified byWearable Device 442 after being placed by the hands of Technician 440,to automatically update a report of the performed task or experiment, tomonitor side effects, or the like.

Referring now to FIG. 5A showing a flowchart diagram of a method, inaccordance with some exemplary embodiments of the disclosed subjectmatter.

On Step 510 a, a determination that a user enters a store may beperformed. In some exemplary embodiments, the user may enter the storeto perform self-service shopping in the store. The determination may beperformed based on an input from the store, such as manual input from aworker in the store, visual input from cameras in the store,geo-location input of the user, or the like.

On Step 520 a, the user may be determined to wear a wearable deviceassociated with the store. In some exemplary embodiments, the wearabledevice may be provided to the user from the store, such as by theretailer, the worker, or the like. Additionally or alternatively, theuser may autonomously collect the wearable device upon entering thestore.

On Step 530 a, a paring action between the wearable device and the usermay be identified. In some exemplary embodiments, the pairing may beperformed between the wearable device and a computing device of theuser, such as via a mobile application. Additionally or alternatively,the pairing may be performed by connecting the wearable device to anaccount of the user associated with the store, to a temporary accountbased on entering debit card details, or the like.

On Step 540 a, a self-service shopping session of the user may beinitiated. In some exemplary embodiments, in response to the pairingaction, self-service shopping session of the user may be enabled orinitiated, such as by opening a gate to the shopping area, enabling theuser to enter the shopping area, or the like.

On Step 550 a, actions performed by a hand of the user may becontinuously monitored by the wearable device. In some exemplaryembodiments, the wearable device may comprise sensors that areconfigured to capture the hand of the user. The input provided by thesensors may be analyzed to determine an action performed by the user andan object the action is performed thereon. Additionally oralternatively, The input provided by the sensors may be analyzed todetermine a physical shopping cart of the user during the self-serviceshopping. It may be noted that the physical shopping cart may changeduring the shopping session, and the visual input may be continuouslyanalyzed to validate the physical shopping cart.

On Step 560 a, a virtual cart of the user may be updated based on theidentified actions and objects. In some exemplary embodiments, thevirtual cart may be updated to include objects identified by thewearable device to be picked by the user and placed in a physicalshopping cart, and to exclude objects removed therefrom.

On Step 660, a check-out may be performed. In some exemplaryembodiments, the check-out may comprise performing a transaction fromthe user to the store based on the content of the virtual cart.

Referring now to FIG. 5B showing a flowchart diagram of a method, inaccordance with some exemplary embodiments of the disclosed subjectmatter.

On Step 510 b, a check-in activity being performed in association with awearable device, may be detected. In some exemplary embodiments, thewearable device may be designed to be worn by a subject. The wearabledevice may be utilized to monitor the subject, and particularly, handactions of the subject.

In some exemplary embodiments, the check-in activity may be performedmanually by the subject, a supervisor thereof, an administrator, or thelike. Additionally or alternatively, the check-in may be performedautomatically when the subject wears the wearable device, when thesubject enters a monitoring area, when a computing device of the subjectconnects to a monitoring device, or the like. Additionally oralternatively, the check-in activity may be associated with apredetermined schedule of the subject, such as shift plans, expectedarrival time to monitored locations, or the like.

On Step 520 b, the subject may be continuously monitored using thewearable device. In some exemplary embodiments, the wearable device maybe configured to monitor hand actions of the subject. The wearabledevice may comprise one or more sensors that are configured to captureat least an interior portion of a hand of the subject, such as thedistal portion of the palm of the hand, the fingers, the area betweenthe fingers and the palm, or the like. The one or more sensors may beconfigured to provide a visual input capturing at least a portion of anobject when being held by the hand. Additionally or alternatively, theone or more sensors may be configured to capture an object grasped byfingers of the subject during grasping thereof.

On Step 530 b, a visual input may be obtained from the sensor located onthe wearable device. In some exemplary embodiments, the visual input maycomprise pictures, videos, or any other visual signals recorded by theone or more sensors, such as ultrasound images, holographic images,laser images, heat maps, or the like. The visual input may comprisetwo-dimensional (2D), three-dimensional (3D) digital images, heat mapspanoramic images, spherical images, or the like. In some exemplaryembodiments, the visual input may be provided after a processing of thesignals captured by the sensor, such as in an imaging process.

In some exemplary embodiments, the wearable device may be configured tobe worn on a wrist of the subject, such as at least one of the one ormore sensors are positioned to face a palm of the hand. The visual inputprovided by the one or more sensors may be configured to comprise atleast a portion of the interior part of the hand, such as the portionbetween the palm and the fingers, a portion of the palm between thethumb and the little finger when being folded, or the like. Additionallyor alternatively, the wearable device may be configured to be worn onthe fingers of the subject, such as the one or more sensors are locatedon interior portions of the fingers. It may be noted that the one ormore sensors may not be configured to capture a full view of theenvironment surrounding the subject, as a view of the one or more sensormay be blocked, at least in part, by the hand of the subject. Such mayenable preserving privacy of the subject and other subjects in thesurrounding area, may reduce the amount of irrelevant data, or the like.It may however, be required that the view captures at least the interiorpart of the hand, such that at least a portion of an object grasped bythe hand is captured by the one or more sensors. Accordingly, at least5%, 10%, 20%, or the like, of the visual input may comprise a view ofthe at least interior portion of the hand, such as the palm, fingers, orthe like.

On Step 540 b, an action performed by the subject and an object theaction is performed thereon, may be identified based on the visualinput. In some exemplary embodiments, the action may be performed by oneor both hands of the subject, or by portions thereof, such as the palms,two or more fingers, or the like. In some exemplary embodiments, theaction may comprise touching the object using one or both hands of thesubject, picking up the object using one or both hands of the subject,releasing the object after being held by one or both hands of thesubject, pressing on the object using one or both hands of the subject,moving the object from a first location to a second location one or bothhands of the subject, unwrapping the object using one or both hands ofthe subject, or the like.

In some exemplary embodiments, the identification of the action and theobject may be performed by analyzing and processing the visual input,using Artificial Intelligence (AI), machine learning techniques,computer vision techniques, image analysis techniques, such as patternrecognition, digital geometry, or the like. Such techniques may utilize2D and 3D object recognition, image segmentation, motion detection, 3Dpose estimation, automatic numbers recognition, barcode scanning, or thelike.

Additionally or alternatively, tracking disposition of the object may besupported by motion sensors or detectors in the wearable device, whichmay be configured to recognize any movement of the wearable device.Additionally or alternatively, an identification of the object may beperformed or supported by barcode scanner.

In some exemplary embodiments, the visual input, comprising pictures,videos or signals recorded by the wearable device, may be sent to andprocessed by a supporting software configured to match these pictures,videos or signals for the purpose of identifying the object and theaction performed thereon. In some cases, the sensor may have a limitedvisibility of the object and will provide partial pictures, videos orsignals thereof. The supporting software may be configured to analyzeand match partial pictures, videos or signals of the item and providethe required output from these partial pictures, videos or signals. Thesupporting software may utilize AI and machine learning to recognize asmany as possible actions made by the subject with each object and toidentify as many as possible items.

Additionally or alternatively, additional input may be utilized toidentify the action and the object, in addition to the visual input,such as input from an accelerator located on the wearable device, or thelike.

On Step 550 b, a responsive action may be performed based on at leastone of the action and the object. It may be noted that determining andperforming the responsive action, may be based on on-device computationsor on off-device computations, such as performed by an edge device, acloud-based server, or the like.

In some exemplary embodiments, the responsive action may comprisedetermining a geospatial location of the object that was identified tobe moved by the subject. A mapping of items in a facility may be updatedto indicate the object is located at the geospatial location. As anexample, mapping of tools in a work location may be continuously updatedbased on determining actions related to moving tools. As anotherexample, stock of merchandise in a store may be continuously monitoredand updated.

Additionally or alternatively, the responsive action may be related tosafety inspection. The identified action may be compared with a safetyrule, such as avoiding mixing materials in a lab, locating an item in adangerous place, or the like. In response to a violation of the safetyrule, the responsive action may comprise issuing a safety alert to thesubject, to a supervisor thereof, or the like. As an example, the safetyrule may be a rule relating to an administration of medicine to apatient, such as the dosage, means of administration, sensitivity todrugs, or the like. The subject may be monitored during treatment ofpatients, and particularly during administration of the medicine. Theviolation of the safety rule may be administering a wrong dosage of themedicine, administering a wrong medicine, performing a wrong procedure,administering the medicine to a wrong patient, or the like.

On Step 560 b, a check-out activity being performed in association withthe wearable device, may be detected.

In some exemplary embodiments, the check-out activity may be performedmanually by a supervisor or an administrator related to the monitoringtask. It may be noted, that the subject may be devoid from manuallyperforming a check-out activity without permission, deactivating thewearable device, or the like. Additionally or alternatively, thecheck-out activity may be performed automatically when the subjectreturns the wearable device to a docketing station, when the subjectexists the monitoring area, when a computing device of the subjectdis-connects from an associated monitoring device, or the like.Additionally or alternatively, the check-out activity may be associatedwith a predetermined schedule of the subject, such as shift plans,expected arrival time to monitored locations or expected departure timestherefrom, or the like. As an example, the check-in and check-outactivities may be automatically performed based on workday schedule ofan employee, based on a smart calendar of the user, or the like.

On Step 570 b, the continuous monitoring may be terminated, in responseto the check-out activity. In some exemplary embodiments, termination ofthe continuous monitoring may be associated with a conclusion responseor action, such as issuing a report, performing automatic check-out andpayment, or the like.

Referring now to FIG. 5C showing a flowchart diagram of a method, inaccordance with some exemplary embodiments of the disclosed subjectmatter.

On Step 530 c, an input from sensors of a wearable device may beobtained. In some exemplary embodiments, the wearable device may be wornby a subject. The input may be obtained from a plurality of sensorslocated on the wearable device, such as cameras, scanners, pressuresensors, or the like. The input may comprise pictures, videos, signals,or the like.

On Step 535 c, the input may be analyzed for the purpose of monitoringor tracking the actions of the hand of the subject.

On Step 540 c, an item with which the hand of the subject made anyaction (or refrained from doing an action) may be identified. In someexemplary embodiments, the identification may be performed by analyzingthe input data, and comparing with objects of items located in themonitored data, with a registry of items, with a list of rules, or thelike.

On Step 545 c, the action made by the hand of the subject may beanalyzed. In some exemplary embodiments, identification of the actionmay be performed based on analysis of the input, such as determiningfeatures in the pictures indicative of a certain action, determining amovement pattern of the hand associated with an action, or the like.Such actions may be: getting close to the item, touching the item,picking the item, moving the item, releasing the item, or the like.

On Step 550 c, AI techniques may be applied to determine the purchase.The AI techniques may be configured to learn the system usage and user'sbehavior in order to determine the action performed by the shopper andwhether the shopper purchased the item. The AI system may furthercollect and store new and existing items and objects pictures, videos orsignals for further learning and training. The AI techniques may beconfigured to improve the system's recognition and identification ofitems and objects, and actions made by the user of the wearable smartretail device.

On Step 560 c, a determination whether the shopper left the monitoredarea may be performed. In some exemplary embodiments, the determinationmay be performed based on location system of a device of the subject,such as a Global Positioning System (GPS) of a device of the user.Additionally or alternatively, the determination may be performed basedon a disconnection between the wearable device and a device associatedwith the monitored area, based on a disconnection between the wearabledevice and a device associated with the user, or the like.

On Step 570 c, the check may be calculated based on the purchased itemswithout the shopper being required to stop at checkout.

Referring now to FIG. 6A showing a flowchart diagram of a method, inaccordance with some exemplary embodiments of the disclosed subjectmatter.

On Step 610, a check-in activity being performed in association with awearable device for self-service shopping, may be detected. In someexemplary embodiments, the wearable device may be designed to be worn bya subject. The wearable device may be utilized to monitor the subject,and particularly, hand actions of the subject, during self-serviceshopping of the subject in a store, such as depicted in FIG. 2. Thecheck-in activity may be associated with the subject commencing aself-service shopping session at the store.

In some exemplary embodiments, the wearable device may be provided tothe subject by the store. The check-in activity may be performedmanually by a retailer or a manger of the store, upon providing thewearable device to the subject, in response to the subject removing thewearable device from a docketing station thereof, upon an activity ofthe subject related to providing a means of payment, or the like.Additionally or alternatively, the wearable device may be a personaldevice of the subject, that may be configured to connect to a system ofthe store. The check-in may be performed automatically when the subjectenters the store, when the wearable device of the subject connects to amonitoring device of the store, or the like. It may be noted that insome cases a preliminary registration prior to a first check-in activitymay be required, such as to update a shopping profile of the subject,creating an account, providing payment method, providing a shoppinglist, or the like.

On Step 620, the subject may be continuously monitored in the storeusing the wearable device (Similar to Step 520 of FIG. 5A). In someexemplary embodiments, the wearable device may be configured to monitorhand actions of the subject. In some exemplary embodiments, the wearabledevice may be synchronized with other monitoring device of the store,such as security cameras, scales, or the like.

On Step 630, a visual input may be obtained from the sensor located onthe wearable device (Similar to Step 530 of FIG. 5A). In some exemplaryembodiments, the visual input provided by the wearable device may beenriched using input from sensors of the stores, such as by providingtemperature information, weights of objects, or the like. Additionallyor alternatively, the visual input may be provided with additional inputfrom other sensors located on the wearable device such as positioningreading of the wearable device, accelerometer readings of the wearabledevice, or the like.

In some exemplary embodiments, the visual input may be analyzed toautomatically determine a real-time shopping cart utilized by thesubject during the shopping in the store. The real-time shopping cartmay be a physical cart, a shopping bag, a personal bag, a tote, one ofthe hands of the subject, or any other container. The real-time shoppingcart may change during the shopping session. As an example, the subjectmay utilize different shopping bags for different types of objects, thesubject may keep one or more objects in her hands before moving to theshopping cart, the subject may put one or more objects in a plastic bagbefore being places in the shopping cart, or the like.

On Step 640, an action related to shopping performed by the subject, andan object from the store that the action is performed thereon, may beidentified based on the visual input (Similar to Step 540 of FIG. 5A).In some exemplary embodiments, the action may be picking up an item forsale, holding an item, returning an item to its initial location,putting the item in the real-time shopping cart, removing the item formthe real-time shopping cart, changing the real-time shopping cart, orthe like.

Additionally or alternatively, a returning action may be identifiedbased on the visual input, based on an input flowing the visual input,such as later in time, proceeding the visual input, or the like. Thereturning action may comprise removing the object from the shoppingcart, returning the object to a location within the store, or the like.The returning action may be indicative of the subject decision not topurchase the object after being determined as item for sale.

Additionally or alternatively, the identification of the action and theobject may be performed using additional input from other sensors on thewearable device, such as a positioning reading of the wearable deviceduring performing the action, accelerometer readings, or the like. Insome exemplary embodiments, a subset of a catalog of items may bedetermined based on the positioning reading of the wearable device, suchas a catalog comprising items located in a respective location withinthe store. A product recognition with respect to the subset of thecatalog of items, may be performed to identify the object.

In some exemplary embodiments, the action may be determined to beassociated with a tampering event during the self-service shoppingsession. The tampering event may be an event aimed at interfering withmonitoring of the hand of the subject using the wearable device. Theaction may comprise removal of the wearable device, blocking view of thesensor, or the like. In some exemplary embodiments, detection of suchactions may be performed based on a behavioral pattern of the subjectthat is indicative of an attempt to prevent monitoring of the hand ofthe subject, such as a behavioral pattern indicating one hand removingthe wearable device from the other hand, moving the hand in certaindirection, or the like. The behavioral pattern may be determined basedon accelerometer readings of the wearable device to determine.

On Step 650, a responsive action related to the shopping process may beperformed based on at least one of the action and the object. In someexemplary embodiments, the responsive action may comprise updating avirtual cart of the subject to include the object picked up by thesubject as a purchased item. In some exemplary embodiments, a content ofthe virtual cart may be displayed to the subject, such as on a mobiledevice of the subject, on a screen on the shopping cart, or the like.The responsive action may further comprise highlighting the object asthe recent item added to the cart, suggesting approval of the item bythe subject, displaying the price of the object, alerting the subject ofsales associated with the object, or the like. The responsive action mayfurther comprise emitting an auditory cue indicating the addition of theobject to the virtual cart, or the like. In some exemplary embodiments,the responsive action may further comprise automatically calculating anupdated check to include the price of the object.

In some exemplary embodiments, the wearable device may be configured toprovide a real-time signal and feedback for the subject, a retailer, orthe like. The real-time signal may comprise identification of the objectand actions made with it. Such real-time signal may be apositive/negative signal, such as green light displayed by wearabledevice to make a positive identification of an item, and a red lightwhich will be displayed when the retail smart wristband identifies thatan item was picked but was unsuccessful with identifying the parametersof the item (unique name or identifier) or the action made with theitem, or the like. Additionally or alternatively, the positive ornegative identification may be displayed using the shopping cart of thesubject, such as via the screen, using LED lights, or the like. Thesignal may be sent in real-time or later to the retailer as well, andmay be used by the retailer, for example, in real time to determine if ashopping cart or a shopping bag should be “qualified” or “disqualified”for continuing the shopping under self-service. Additionally oralternatively, the objects which the system recognized as released inthe shopping cart or bag, may be listed in a designated mobile app. Thesubject may be able to check in real-time if the system's list isaccurate, and to avoid the inconvenience caused by leaving the storewith unlisted items.

Additionally or alternatively, in response to determining a returningaction, the responsive action may comprise updating the virtual cart ofthe subject to exclude the object. The responsive action may furthercomprise emitting an auditory cue indicating the removal of the objectto the virtual cart, updating the check to exclude the price of theobject, suggesting alternative items to the subject, or the like.

Additionally or alternatively, in response to detecting a tamperingevent, the responsive action may comprise performing an anti-tamperingaction, such as issuing an alert, ending the shopping session,indicating the associated object as a purchased item, or the like. Itmay be noted that the wearable device is configured to perform detectionof such tampering events, only during the self-service shopping session.The subject may be able to perform the tampering event after theself-service shopping session ends without resulting in theanti-tampering action.

Additionally or alternatively, the wearable device may be utilized toremove a theft detection tag coupled with the object. The responsiveaction may comprise indicating the object as purchased, ensuring paymentfor the object, or the like.

On Step 660, a check-out activity being performed in association withthe wearable device, may be detected. (Similar to Step 560 b of FIG.5B). In some exemplary embodiments, the check-out activity may beindicative of finishing the self-service shopping, such as by exitingthe store, returning the wearable device, or the like. In some exemplaryembodiments, the continuous monitoring may be terminated, in response tothe check-out activity. Additionally or alternatively, the check-outactivity may be associated with an inspection activity before allowingthe subject to leave the store, such as a manual inspection performed bythe retailer, an automatic inspection performed by a scanning machine ofthe store, an indication from the wearable device (such as via an outputcomponent thereof) that the virtual cart is validated, or the like.

On Step 670, a transaction may be performed based on content of thevirtual cart of the subject. In some exemplary embodiments, a check maybe calculated based on the items that were identified to be picked, putin the shopping cart and updated in the virtual cart. A transactionmatching the check may be performed. In some exemplary embodiments, thetransaction may be required to be approved the subject prior to beingexecuted. In some exemplary embodiments, the continuous monitoring maybe terminated, in response to the check-out activity. In some exemplaryembodiments, the continuous monitoring may be terminated.

Referring now to FIG. 6B showing a flowchart diagram of a method, inaccordance with some exemplary embodiments of the disclosed subjectmatter.

On Step 610 b, a check-in activity being performed in association with awearable device for on-line shopping service, may be detected. In someexemplary embodiments, the wearable device may be designed to be worn bya picker responsible of manual fulfilment of shopping orders of one ormore customers. The wearable device may be utilized to monitor thepicker during the fulfilment of the shopping order in a store, such asdepicted in FIG. 2.

In some exemplary embodiments, the wearable device may be utilized bythe picker for multiple orders simultaneously. Each check-in activitymay be associated with a different customer. The check-in activity maycomprise connecting to a list of items selected by the customer,connecting to an online chat with the customer, or the like. Thecheck-in activity may be performed manually by the picker. The pickermay be enabled to switch between accounts of different customers duringthe shopping session.

On Step 620 b, the picker may be continuously monitored in the storeusing the wearable device (Similar to Step 620 of FIG. 6A).

On Step 630 b, a visual input may be obtained from the sensor located onthe wearable device (Similar to Step 630 of FIG. 6A).

On Step 640 b, an action related to the fulfilment of the shopping orderperformed by the picker, and an object from the store that the action isperformed thereon, may be identified based on the visual input (Similarto Step 640 of FIG. 6A). In some exemplary embodiments, the action maybe picking up an object, holding an object, returning an object to itsinitial location, placing the object in a shopping cart or a toteassociated with the shopping order of the customer, removing the objectform the shopping cart, or the like.

On Step 650 b, a responsive action related to the picking process (e.g.the fulfillment of the shopping order of the customer) may be performedbased on at least one of the action and the object. In some exemplaryembodiments, the responsive action may comprise identifying acorresponding item to the object in the list of items, and marking thecorresponding item as fulfilled. Additionally or alternatively, theresponsive action may comprise identifying a mismatch between the objectand the list of items, and accordingly alerting the picker of themismatch.

On Step 660 b, a check-out activity being performed in association withthe wearable device, may be detected. (Similar to Step 660 of FIG. 6A).In some exemplary embodiments, the check-out activity may be indicativeof finishing the order of the associated customer. In some exemplaryembodiments, the continuous monitoring may be terminated, in response tothe check-out activity. Additionally or alternatively, the continuousmonitoring may be continued with respect to other orders fulfilment ofother customers.

On Step 670 b, a transaction may be performed based on content of thevirtual cart of the customer. In some exemplary embodiments, a check maybe calculated based on the items that were identified to be picked, putin the shopping cart and updated in the virtual cart.

Referring now to FIG. 7 showing a block diagram of a system, inaccordance with some exemplary embodiments of the disclosed subjectmatter.

In some exemplary embodiments, a System 700 may be utilized to manage aself-service shopping of a User 705 in a store, or online-shoppingfulfillment for customers by User 705, or the like. Additionally oralternatively, similar applications of System 700 may be utilized forother facilities to perform monitoring of hand actions of users, such asin health-care systems to monitor action of health-care staff members,in airplanes to monitor actions of pilots, in augmented reality videogames to monitor actions of players, or the like.

In some exemplary embodiments, System 700 may comprise a plurality ofWearable Devices 710 each of which is being worn on a hand of user suchas User 705. Each Wearable Device 710 may be configured to be worn onthe hand of User 705, in a manner enabling capturing interior portionthereof, such as on a wrist of User 705, on fingers of User 705, on ahand palm of User 705, or the like. Wearable Device 710 may beconfigured to be utilized to identify items grabbed by the hand of User705 and moved to or from a physical shopping tote of User 705.

In some exemplary embodiments, Wearable Device 710 may comprise a VisualSensor 712. Visual Sensor 712 may be configured to continuously capturean interior portion of the hand of User 705. Wearable Device 710 may beconfigured to provide visual input captured by Visual Sensor 712 to beutilized to identify activity performed by the hand of User 705, such asan action performed by the hand, an object upon which the action isperformed, or the like. Visual Sensor 712 may comprise a single lens,one or more lenses, or the like. Visual Sensor 712 may be configured tocapture pictures, videos, signals, a combination thereof, or the like.In some exemplary embodiments,

In some exemplary embodiments, Wearable Device 710 may comprise aCommunication Unit 714. Communication Unit 714 may be configured toconnect Wearable Device 710 to a controller external thereto, such as toa mobile Device 720 of User 705, Store Unit 730, to Server 740, or thelike. Wearable Device 710 may be automatically activated when connectedto Store Unit 730, such as based on connecting to a Wi-Fi network in thestore associated with Store Unit 730, using an activation interfaceassociated with Store Unit 730, based on the location readings ofWearable Device 710 being conformed with location of Store Unit 730, orthe like. Similarly, Wearable Device 710 may be de-activated whenleaving the store, such as based on dis-connecting from Store Unit 730,based on store Unit identifying that User 705 left the store, or thelike.

In some exemplary embodiments, Wearable Device 710 may be associatedwith an application of a computing Device 720 of User 705, such as amobile app, or the like. The mobile app may be a standalone native app,a feature embedded in or hosted by third party app(s), or the like. User705 may receive data associated with the shopping session to Device 720,provide feedback, or the like. The data may be provided in real time orpost actions. In some exemplary embodiments, the data may be displayedon a screen of Device 720, using the designated application or the like.As an example, Device 720 may be utilized to display a Virtual CartDisplay 722 for User 705, upon initiating a self-shopping session,indicating the items shopped thereby. Additionally or alternatively,Device 720 may be utilized to display a Shopping List 724 for User 705.

In some exemplary embodiments, System 700 may comprise a Server 740.Server 740 may be configured to support the monitoring andidentification of hand actions of users in the store, such as User 705,to perform respective responsive actions, to issue output to User 705 orto Store Unit 730. or the like.

In some exemplary embodiments, activation and de-activation of WearableDevice 710 may be performed automatically by an Activation Module 745 ofServer 740. Wearable Device 710 may be devoid of a de-activationinterface for User 705. The activation of Wearable Device 710 may beperformed in response to identifying a check-in activity associated withUser 705, such as a connection from Wearable Device 710 to Store Unit730, an indication from Activation Module 745 that Wearable Device 710is collected by User 705, based on a pairing between Wearable Device 720and Store Unit 730, or the like. Similarly, the deactivation of WearableDevice 710 may be performed in response to determining a check-outactivity associated with User 705.

In some exemplary embodiments, Server 740 may comprise a Control Module750 configured to analyze input obtained from Wearable Device 710 toidentify an action performed by the hands of User 705 and an objectwhich upon the action is performed. Control Module 750 may be configuredto identify items or objects with which the hand of User 705 made anyaction, or refrained from doing an action therewith. Additionally oralternatively, the action may be associated with modifying content of aphysical shopping tote of User 705. Control Module 750 may be configuredto recognize when the hand of User 705 is getting close to an item,picking an item, holding an item (as an example, while the object staysconstant at the hand), moving an item (as an example, background picturechanged), releasing an item, or the like.

In some exemplary embodiments, Server 740 may comprise a CatalogDatabase 780 retaining visual representations of items in the store.Control Module 750 may be configured to recognize the object which uponthe hand of User 705 performs the action based on Catalog Database 780.Catalog Database 780 may be retained by System 700. Control Module 750may be configured to compare and match objects identified in the inputwith objects of items stored in Catalog Database 780. Prior toactivating the solution, all items which may be sold in the store may bepictured from different angels, categorized and stored in CatalogDatabase 780. When Control Module 750 recognizes that the hand(s) ofUser 705 made an action or got close to an item, the picture, video orsignal of the item may be matched with Catalog Database 780, andidentified thereof (or not identified, if such item is not listed in thedatabase or matching has not succeeded). Additionally or alternatively,Catalog Database 780 may comprise a plurality of partial databases foreach store. The partial database may comprise items that are known to bein the store in a certain location. In order to speed the identificationof an item, instead of searching the entire database each time, only theitems, which the service is aware of being located in the store at thecertain location, or located next to the location of Wearable Device710, may be searched. In case that Control Module 750 does not find theitem in that partial database, Control Module 750 may search the wholeCatalog Database 780 for that item. Control Module 750 may be configuredto know what items are located in a certain stores based on an inventorylist obtained from the User 705, Store Unit 730, or the like.Additionally or alternatively, Control Module 750 may be configured toknow what items are located in a certain store based on informationobtained from wearable device used by the retailer's worker uponarranging the store. Each item that is located in the store may beidentified and listed in the partial database of the store.

Additionally or alternatively, Control Module 750 may be configured toidentify parameters of the item, such as type, category, name, shape,size, or the like. Such parameters may be identified based on dataretained in Catalog Database 780, or other databases. In some exemplaryembodiments, Control Module 750 may be configured to apply machinelearning techniques, classification techniques, image processingtechniques, AI techniques, or the like, in order to identify the objectand the action performed thereon, to learn behavior of User 705,shopping habits thereof, or the like. Control Module 750 may beconfigured to collect and store information related to new or existingitems and objects, such as pictures, videos, signals, classifications,or the like, in Catalog Database 780. Control Module 750 may beconfigured to improve the recognition and identification of items andobjects of System 700, actions made by User 705, or the like.

Additionally or alternatively, Control Module 750 may be configured toobtain visual input of the content of the physical shopping tote, suchas from Wearable Device 710, or other visual sensors of the storeassociated with Store Unit 730, sensors of User Device 720, or the like.Control Module 750 may be configured to determine a discrepancy betweencontent of Virtual Cart 760 and the content of the physical shoppingtote, such as based on identifying the items in the physical shoppingtote and comparing the identified items to the items listed in inVirtual Cart 760. Control Module 750 may be configured to perform aresponsive action in response to the determined discrepancy, such as bymarking Virtual Cart 760 as invalidated, updating Virtual Cart 760 basedon the visual input of the content of the physical shopping tote, or thelike.

Additionally or alternatively, Control Module 750 may be configured todetermine a mapping of geo-spatial locations of items in the store.Control Module 750 may be configured to identify a placement location ofeach object moved by User 705 or any other user, such as a worker in thestore, and update the mapping to indicate a location of the object basedon the placement location.

In some exemplary embodiments, Control Module 750 may be configured todetermine a responsive action based on the action or the object. ControlModule 750 may be configured to update a Virtual Cart 760 in response tothe identification of the action by the user, such as adding and item,removing an item, or the like. Virtual Cart 760 may indicate a list ofitems shopped by User 705. Virtual Cart 760 may be automatically updatedbased on items moved to and from the physical shopping tote of User 705,such as by adding items to Virtual Cart 760 based on items picked up andput into the physical shopping tote of User 705 and removing items fromVirtual Cart 760 based on items removed from the physical shopping toteof User 705. Virtual Cart Display 722 on Device 720, may beautomatically updated in response to updating Virtual Cart 760.

In some exemplary embodiments, Control Module 750 may be configured toissue an output to User 705. The output may be issued to Device 720 ofUser 705, such as by displaying the content of Virtual Cart 760 to User705 using Device 720 (Virtual Cart Display 722), such as issuing anaudio alert using a Speaker 718 on Wearable Device 710, using a LEDlight bulb 719 on Wearable Device 710, or any other visual output, toprovide an indication of an addition of an item to or removal of an itemfrom Virtual Cart 760.

In some exemplary embodiments, System 700 may comprise a Payment Module755 configured to manage payments and transactions associated with theself-service shopping. Control Module 750 may be configured to invokePayment Module 755 in response to a check-out activity of User 705.Control Module 750 may be configured to perform a transaction based on alist of shopped items of the user as indicated in Virtual Cart 760.

Additionally or alternatively, Server 740 may comprise a TamperingDetection Module 765. Tampering Detection Module 765 may be configuredto monitor and detect a tamper event during a shopping session of User705. Tampering Detection Module 765 may be configured to avoidmonitoring user activity outside the shopping session. TamperingDetection Module 765 may be configured to determine an attempt toperform a tampering event by User 705. Tampering Detection Module 765may be configured to deduce information from gyroscope and accelerometerreadings, such as to identify attempts by User 705 to block the view ofWearable Device 710, a behavioral pattern indicative of a tamperingevent, or the like. As an example, based on the sensor readings, it maybe possible to identify that the shopper is putting his hand behind hisback in an unnatural manner. As another example, the behavioral patternof the subject determined based on the accelerometer readings may beindicative of one hand removing Wearable Device 710 from the other hand,moving the hand in certain direction, or the like. Additionally oralternatively, Tampering Detection Module 765 may be associated with aTampering Detector 716 located on Wearable Device 710. TamperingDetector 716 may be configured to monitor and detect a tampering eventduring the self-shopping session of user 705. Tampering Detector 716 maybe configured to identify an attempt to remove Wearable Device 710,blocking view of Sensor 712 by the hand or other object, an attempt todisconnect Wearable Device 710 from Store Unit while still located inthe store such as by covering Wearable Device 710 with anelectromagnetic wave insulating material, an attempt to disturb themonitoring such as by entering a Faraday shield used to blockelectromagnetic fields, or the like. It may be noted that TamperingDetector 716 may operate independently from Tampering Detection Module765, without Server 740 comprising Tampering Detection Module 765, orthe like. Tampering Detection Module 765 may be configured to perform ananti-tampering action in response to detecting the tampering event, suchas issuing an alert to Store Unit 730.

It may be noted that System 700 may be configured to operate also incrowded stores, having a ratio between number of people in the store anda floor size of the store in squared meters above 1:10, above 1:20,above 2:10, or the like. As the visual input from each user is obtainedindependently from other users, while on user is not supposed to blockthe visual input capturing the hand actions of the other users. Theanalysis performed on one user may also be independent from the analysisperformed for other users. As an example, a cashier free store,utilizing in-store cameras with an area of 400 square meters, mayfunction only with fewer than 20 shoppers inside, as one shopper mayblock the view for the cameras for observing other shoppers. However,System 700 may operate with any number of shoppers the store canregularly handle, such as above 40 shoppers, above 80 shoppers, above100 shoppers, or the like, as the analysis performed by System 700 isbased on visual input from each shopper, without affecting the visualinput provided by other shoppers in the store.

Additionally or alternatively, Wearable Device 710 may be configured tobe utilized for manual fulfillment of a shopping order of a customer byUser 705. User 705 may be a picker tasked with picking items to fulfillthe shopping order of the customer. The shopping order may comprise aList 724 of items selected by the customer and transmitted to Device 720of User 705. Control Module 750 may be configured to identify a pickingcaptured by Wearable Device 710, such as picking up an object andplacing the object in a tote associated with the shopping order of thecustomer. Control Module 750 may be configured to identify acorresponding item to the item in the shopping order (e.g. in List 724)and mark the corresponding item as fulfilled. In response to adetermination that the shopping order is fulfilled, Control Module maybe configured to invoke Payment Module 755 to enable a transaction fromthe customer to Store Unit 730, based on the fulfilled shopping order,or a portion thereof determined as fulfilled.

The present invention may be a system, a method, and/or a computerprogram product. The computer program product may include a computerreadable storage medium (or media) having computer readable programinstructions thereon for causing a processor to carry out aspects of thepresent invention.

The computer readable storage medium can be a tangible device that canretain and store instructions for use by an instruction executiondevice. The computer readable storage medium may be, for example, but isnot limited to, an electronic storage device, a magnetic storage device,an optical storage device, an electromagnetic storage device, asemiconductor storage device, or any suitable combination of theforegoing. A non-exhaustive list of more specific examples of thecomputer readable storage medium includes the following: a portablecomputer diskette, a hard disk, a random access memory (RAM), aread-only memory (ROM), an erasable programmable read-only memory (EPROMor Flash memory), a static random access memory (SRAM), a portablecompact disc read-only memory (CD-ROM), a digital versatile disk (DVD),a memory stick, a floppy disk, a mechanically encoded device such aspunch-cards or raised structures in a groove having instructionsrecorded thereon, and any suitable combination of the foregoing. Acomputer readable storage medium, as used herein, is not to be construedas being transitory signals per se, such as radio waves or other freelypropagating electromagnetic waves, electromagnetic waves propagatingthrough a waveguide or other transmission media (e.g., light pulsespassing through a fiber-optic cable), or electrical signals transmittedthrough a wire.

Computer readable program instructions described herein can bedownloaded to respective computing/processing devices from a computerreadable storage medium or to an external computer or external storagedevice via a network, for example, the Internet, a local area network, awide area network and/or a wireless network. The network may comprisecopper transmission cables, optical transmission fibers, wirelesstransmission, routers, firewalls, switches, gateway computers and/oredge servers. A network adapter card or network interface in eachcomputing/processing device receives computer readable programinstructions from the network and forwards the computer readable programinstructions for storage in a computer readable storage medium withinthe respective computing/processing device.

Computer readable program instructions for carrying out operations ofthe present invention may be assembler instructions,instruction-set-architecture (ISA) instructions, machine instructions,machine dependent instructions, microcode, firmware instructions,state-setting data, or either source code or object code written in anycombination of one or more programming languages, including an objectoriented programming language such as Smalltalk, C++ or the like, andconventional procedural programming languages, such as the “C”programming language or similar programming languages. The computerreadable program instructions may execute entirely on the user'scomputer, partly on the user's computer, as a stand-alone softwarepackage, partly on the user's computer and partly on a remote computeror entirely on the remote computer or server. In the latter scenario,the remote computer may be connected to the user's computer through anytype of network, including a local area network (LAN) or a wide areanetwork (WAN), or the connection may be made to an external computer(for example, through the Internet using an Internet Service Provider).In some embodiments, electronic circuitry including, for example,programmable logic circuitry, field-programmable gate arrays (FPGA), orprogrammable logic arrays (PLA) may execute the computer readableprogram instructions by utilizing state information of the computerreadable program instructions to personalize the electronic circuitry,in order to perform aspects of the present invention.

Aspects of the present invention are described herein with reference toflowchart illustrations and/or block diagrams of methods, apparatus(systems), and computer program products according to embodiments of theinvention. It will be understood that each block of the flowchartillustrations and/or block diagrams, and combinations of blocks in theflowchart illustrations and/or block diagrams, can be implemented bycomputer readable program instructions.

These computer readable program instructions may be provided to aprocessor of a general purpose computer, special purpose computer, orother programmable data processing apparatus to produce a machine, suchthat the instructions, which execute via the processor of the computeror other programmable data processing apparatus, create means forimplementing the functions/acts specified in the flowchart and/or blockdiagram block or blocks. These computer readable program instructionsmay also be stored in a computer readable storage medium that can directa computer, a programmable data processing apparatus, and/or otherdevices to function in a particular manner, such that the computerreadable storage medium having instructions stored therein comprises anarticle of manufacture including instructions which implement aspects ofthe function/act specified in the flowchart and/or block diagram blockor blocks.

The computer readable program instructions may also be loaded onto acomputer, other programmable data processing apparatus, or other deviceto cause a series of operational steps to be performed on the computer,other programmable apparatus or other device to produce a computerimplemented process, such that the instructions which execute on thecomputer, other programmable apparatus, or other device implement thefunctions/acts specified in the flowchart and/or block diagram block orblocks.

The flowchart and block diagrams in the Figures illustrate thearchitecture, functionality, and operation of possible implementationsof systems, methods, and computer program products according to variousembodiments of the present invention. In this regard, each block in theflowchart or block diagrams may represent a module, segment, or portionof instructions, which comprises one or more executable instructions forimplementing the specified logical function(s). In some alternativeimplementations, the functions noted in the block may occur out of theorder noted in the figures. For example, two blocks shown in successionmay, in fact, be executed substantially concurrently, or the blocks maysometimes be executed in the reverse order, depending upon thefunctionality involved. It will also be noted that each block of theblock diagrams and/or flowchart illustration, and combinations of blocksin the block diagrams and/or flowchart illustration, can be implementedby special purpose hardware-based systems that perform the specifiedfunctions or acts or carry out combinations of special purpose hardwareand computer instructions.

The terminology used herein is for the purpose of describing particularembodiments only and is not intended to be limiting of the invention. Asused herein, the singular forms “a”, “an” and “the” are intended toinclude the plural forms as well, unless the context clearly indicatesotherwise. It will be further understood that the terms “comprises”and/or “comprising,” when used in this specification, specify thepresence of stated features, integers, steps, operations, elements,and/or components, but do not preclude the presence or addition of oneor more other features, integers, steps, operations, elements,components, and/or groups thereof.

The corresponding structures, materials, acts, and equivalents of allmeans or step plus function elements in the claims below are intended toinclude any structure, material, or act for performing the function incombination with other claimed elements as specifically claimed. Thedescription of the present invention has been presented for purposes ofillustration and description, but is not intended to be exhaustive orlimited to the invention in the form disclosed. Many modifications andvariations will be apparent to those of ordinary skill in the artwithout departing from the scope and spirit of the invention. Theembodiment was chosen and described in order to best explain theprinciples of the invention and the practical application, and to enableothers of ordinary skill in the art to understand the invention forvarious embodiments with various modifications as are suited to theparticular use contemplated.

What is claimed is:
 1. A system for a fulfillment of orders in afulfillment site, comprising: a plurality of wearable devices; a controlmodule; a catalog database retaining visual representations of items inthe fulfillment site; and a payment module; wherein each of saidplurality of wearable devices is being worn on a hand of a differentpicker and configured to obtain visual input of activity performed bythe hand of the different picker, wherein each picker is tasked withpicking items to fulfill an order of a different customer; wherein saidcontrol module is configured to identify a picking action that comprisespicking up the object and placing the object in a tote associated withthe order of the customer, wherein the picking action is captured by thewearable device; and wherein said control module is configured toidentify a corresponding item to the item in the order and mark thecorresponding item as fulfilled, wherein the identification of thecorresponding item is based on said catalog database.
 2. The system ofclaim 1, wherein the fulfillment site is selected from a list consistingof: a store, a pharmacy, a medical center, a warehouse, a fulfillmentcenter, and a factory.
 3. The system of claim 1, wherein said controlmodule is configured to invoke a payment module in response to adetermination that the order is fulfilled.
 4. A method comprising:obtaining a visual input from a sensor located on a wearable device,wherein the wearable device is worn on a hand of a picker, wherein thesensor is configured to capture at least an interior portion of a handof the picker, wherein the picker is tasked with picking items tofulfill an order of a customer, wherein the order comprises a list ofitems; determining, based on the visual input, that the picker picked upan item and placed the item in a tote associated with the order of thecustomer, wherein said determining that the picker picked up the itemand placed the item comprises identifying the item; and in response tosaid determining: performing a fulfillment-related action.
 5. The methodof claim 4, wherein said obtaining and said determining are performedduring a collection session, wherein during the collection session,continuously monitoring the picker using the visual sensor, wherein saidcontinuously monitoring the picker comprises said obtaining the visualinput.
 6. The method of claim 5 further comprises: in response toidentification of a first activity, determining that the collectionsession begins; and in response to identification of a second activity,terminating the collection session.
 7. The method of claim 4, whereinthe interior portion of the hand comprises a distal portion of a palm.8. The method of claim 4, wherein the wearable device is configured tobe worn on a wrist of the picker, whereby positioning the sensor to facea palm of the hand.
 9. The method of claim 4, wherein said identifyingthe item is performed based on the visual input and based on apositioning reading of the wearable device.
 10. The method of claim 9further comprises: determining, based on the positioning reading of thewearable device, a subset of a catalog of items; and performing productrecognition to identify the item, wherein the product recognition isperformed with respect to the subset of the catalog of items.
 11. Themethod of claim 4, wherein the wearable device comprises a firstwearable component that comprises a first sensor and a second wearablecomponent that comprises a second sensor, wherein the first wearablecomponent is configured to be worn on the hand of the picker, whereinthe second wearable component is configured to be worn on a second handof the picker, wherein the first sensor is configured to capture atleast the interior portion of the hand of the picker, wherein the secondsensor is configured to capture at least an interior portion of thesecond hand of the picker, wherein the sensor comprises the first sensorand the second sensor, wherein the visual input comprises input from thefirst sensor and from the second sensor.
 12. The method of claim 11,wherein said identifying the item is performed uniformly on acombination of the inputs from the first and second sensors.
 13. Themethod of claim 11, wherein said identifying the item comprisesidentifying the item as a single item observed from two differentviewpoints by the first and second sensors.
 14. The method of claim 4,wherein said performing a fulfillment-related action comprises: matchinga corresponding item in the list of items to the item; and marking thecorresponding item as fulfilled.
 15. The method of claim 4, wherein saidperforming a fulfillment-related action comprises: identifying amismatch between the item and the list of items; and alerting the pickerof the mismatch.
 16. The method of claim 4, wherein the picker is taskedwith picking items to fulfill a plurality of orders of a plurality ofcustomers simultaneously during a single collection session; whereinsaid determining that the picker picked up the item and placed the itemcomprises identifying the customer associated with the tote, in whichthe item is placed.
 17. The method of claim 4, wherein said identifyingthe item is performed based on the list of items.
 18. The method ofclaim 4, wherein said identifying the item is performed based on anattempt to match the item with items appearing on the list of items. 19.The method of claim 4, wherein the fulfillment-related action comprisesadding the item to a virtual cart; and wherein the method furthercomprises: obtaining a second visual input from the sensor of thewearable device; determining, based on the visual input, that the pickerremoved the item from the tote; and in response to said determining:removing the item from the virtual cart.
 20. A wearable devicecomprises: a visual sensor; and a communication unit configured toconnect said wearable device to a controller external to the wearabledevice; wherein said wearable device is configured to be worn on a handof a picker, wherein the picker is tasked with a fulfillment task,wherein the fulfillment task comprises picking one or more items tofulfill an order of a customer; wherein said visual sensor is configuredto continuously capture an interior portion of the hand of the picker;wherein the wearable device is configured to provide images captured bythe visual sensor to be utilized to identify a picking action performedby the hand of the picker and to identify an object upon which thepicking action is performed, wherein the picking action comprisespicking up the object and placing the object in a tote used forfulfilling the order of the customer.
 21. The wearable device of claim20, wherein the wearable device is configured to be in communicationwith a server that provides the picker with the fulfillment task. 22.The wearable device of claim 20, wherein the wearable device isconfigured to be in communication with a server, wherein the server isconfigured to activate and deactivate monitoring by the wearable deviceusing the visual sensor for a duration of collection session, whereinthe wearable device is configured to continuously monitor the pickerusing the visual sensor during the collection session.